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UTR: A Different, 
More Durable Model 


INTRODUCTION 


Te report shares findings from three longitudinal studies of the Urban Teacher Residency (UTR) Project, a partnership 
between New Visions for Public Schools, Hunter College, and the New York City Department of Education. From 2009 to 
2014, with funding from the US Department of Education’s Teacher Quality Partnership (TQP) Program, UTR placed over 
150 new English, mathematics, science, and special education teachers in some of the city’s highest-need secondary 
schools. In 2014, partners were awarded a three-year supplemental grant in support of the research shared here. 


Funding for the longer-term research required evidence of shorter-term impact. For UTR, both internal program data and 
external evaluation data had shown promising results: Each cohort of UTR residents, selectively chosen, increasingly 
diverse, had completed all program requirements with only modest attrition, spending a year in a high-needs school 
under the guidance of a trained mentor, meeting benchmarks on a set of program assessments that included a 
demonstratration of effective practice, completing Hunter coursework, passing required New York state licensure exams, 
and obtaining jobs in high-needs schools. 


Survey and focus group data, gathered at the end of each UTR cohort’s clinical year by Rockman et al, UTR’s external 
evaluator, confirmed that residents were entering classrooms confident in their skills and knowledge—and in their 
decision to become teachers. Year after year, Rockman’s annual analyses of student achievement also showed that 
students taught by UTR teachers were performing as well as, often better than their peers. Perhaps most important, 
early retentention data indicated that UTR were staying, at rates that surpassed city-wide figures. 


A selective admissions process, a skill- and confidence-building full year in the classroom, built-in accountability —all 
seemed to have paid off: UTR was increasing the numbers of teachers— effective teachers —in the pipeline. 


What the early data couldn’t confirm was whether the benefits were long-term, even increasing as teachers gained 
experience. To explore those questions, Rockman, in collaboration with UTR partners, designed three supplemental 
studies. 


[HE TAREE STUDIES 


| he individual sections of this report explain the rationale and research design for the three studies. The summaries 
below describe the studies, and the questions that guided them, in brief. 


Student Achievement 

Student achievement warranted a separate study for two main reasons: first, at the end of UTR’s five years, UTR’s 
successive cohorts had been teaching four years or less, and a longer time frame was needed to gauge impact. Second, 
student performance remains an important indicator of teachers’ 

effectiveness, and standardized test scores provide a common metric Do UTR teachers improve 

for a quasi-experimental design. student achievement? 


Special Education 

Special education merited its own study because close to half of all UTR residents were special educators. It also 
required a different kind of study: achievement data wasn’t the right metric for special ed students, and a more 
qualitative study with site visits and interviews, anchored by analyses of 


school performance data, could probe what UTR special educators What impact do UTR 
brought to schools—especially important as NYC DOE schools were special educators have on 
adjusting to major changes in district special education policies. students and schools? 


Teacher Retention 

At the end of UTR’s five-year funding period, the first cohort of teachers had been teaching only four years—the term of 
their program commitment. It was important to look beyond that, to see whether UTR teachers were staying—in high- 
need NYC DOE high schools—and how retention patterns compared to those of teachers trained in other programs. A 
separate study could also explore factors linked to teachers’ decisions 

to stay, change schools, or leave the profession. Do UTR teachers stay, 


and why or why not? 


[HE KEY FANDINGS 


Be about results and their significance appear in a Summary of Findings that accompanies each study. The 
bullets and conclusions below share the key takeaways from all three studies. 


Student Achievement 

= UTR teachers have a positive impact on student achievement. NY State Regents exam scores across all 
subject areas showed that UTR-trained teachers’ students performed as well as or better than peers taught by 
teachers trained in other programs. We compared scores using analysis of covariance (ANCOVA), controlling for 
gender, English language learner status, special education status, ethnicity, and prior achievement. 


=" The differences in performance that were statistically significant favor UTR. In 27 comparisons of Regents 
scores where differences between students taught by UTR- and non-UTR-trained teachers were statistically 
significant, the UTR group’s performance was higher 89% of the time. 


UTR benefits increase over time in some subjects. For Geometry, Algebra 2 — Trigonometry, and Earth Science 
become stronger as teachers gain classroom experience: a positive gap between UTR vs non-UTR was more 
evident in the group with three or more years or teaching experience. 


Special Education 


UTR special educators have a positive impact on factors that lead to student success. 11th and 12th 
grade special education students in schools with UTR-trained special educators had higher attendance rates and 
earned more credits than special education students in schools with no UTR-trained teachers. 


UTR special educators can benefit students of color. Comparative analyses of attendance, credit 
accumulation, and graduation data suggested a similar predictive value or positive impact of UTR special educators 
on Hispanic and African-American students. 


UTR training equipped residents not just to acclimate to urban schools but also contribute to them. 
Teachers and administrators in placement schools praised this skillset around data and intervention, and credited 
residents with helping them meet the needs of special education students. 


UTR special educators help schools manage change. UTR special educators began teaching as NYC schools 
were adjusting to new policies, and feedback, especially from principals, indicated that UTR special educators 
helped schools adapt and carry out changes in ways that supported both students and teachers. 


Retention 


UTR teachers are staying in the classroom, at an overall retention rate of 91%. UTR graduates had a lower 
rate of attrition—by half—than other new NYC DOE high school teachers. Retention rates decline slightly over time, 
but, after six years, close to three-fourths of UTR’s first cohort are still teaching. 


Retention rates are highest among special educators, at 93%. These rates exceed rates among special 
educators city-wide, and rates for special educators trained in traditional and other alternative certification programs. 
Mobility data also indicate that special educators are less likely to change schools. 


Background characteristics associated with retention include ethnicity and prior schooling. Among UTR 
teachers, teachers of color have comparable or higher retention rates, and as do those with bachelors’—as 
opposed to advanced— degrees. 


Residents’ perceptions of efficacy and preparedness are related to the likelihood that they will remain in 
the classroom. Residents’ self-assessments of classroom management skills are a statistically significant factor 


when analyzing retention status. 


A wide range of factors that merit further investigation can influence retention. Hiring school culture and 
demographics may be factors, as are program service requirements influence teacher retention. Of the UTR and 
MASTER graduates who are no longer teaching, about a quarter left after the four-year commitment, compared to 
almost half of Teach for America teachers leaving after fulfilling the two-year service requirement. 


CONCLUSIONS 


1. studies were designed to gauge UTR’s success in reaching two key goals—improving student achievement 
and teacher retention—and to gain a broader perspective on the longer-term impact of the UTR model. Our findings are 
based on multiple data sources, including external evaluation survey, focus group, and interview data; program data from 
New Visions and Hunter College; annual student achievement data, including comparison group data; publicly available 
school performance and climate data from the NYC DOE; and reports from the New York City Independent Budget 
Office. We also consulted research in the field to see how UTR fared compared to other models, and what these studies 
might contribute to conversations about teacher preparation and quality. 


Overall, our findings portray a teacher preparation model that, thus far, stands the test of time, and helps build the body 
of evidence called for in a recent Bellwether Education Partners report urging accountability for teacher residences. Our 
Achievement Study, based on standardized Regents test scores from students taught by UTR teachers with between 
one and seven years’ experience, show UTR-taught students performing as well as their peers, in some cases 
significantly higher. Findings also indicate that some UTR benefits become stronger over time. 


The Retention Study similarly portrays a model with staying power. Retention rates among UTR graduates are high, and 
mobility rates are low. Rates decline slightly the longer teachers are in the classroom, but, after six years, close to three- 
fourths of UTR’s first cohort are still teaching. Compared to peers prepared through other alternative certification 
pathways, UTR-trained teachers have lower rates of attrition by half or more. 


Much school research, especially longitudinal research, involves a host of confounding variables, and these studies were 
no different. What may be most compelling is the consistency: the longitudinal achievement results mirror the results 
from our annual analyses, which, each year, show an overall advantage to being taught by a UTR-trained teacher 
compared to a non-UTR trained teacher. Findings also held steady in annual results for residents trained in the 
Mathematics and Science Teacher Education Residency (MASTER) project, a joint, NSF-funded effort for which UTR 
partners adapted the UTR model.” Findings from other alternative programs also indicate a positive impact, but chart 
peaks and valleys, where teachers either do less well than peers in their first year, but gradually show positive results, or 
have an initial positive impact that plateaus over time.° 


The consistency in UTR findings may reflect the accountability built into the UTR model. Partners routinely assemble 
data, through the New Visions suite of performance assessments and placement and retention tracking, and Hunter’s 


1 
Ashley LiBetti and Justin Trinidad, “Trading Coursework for Classroom: Realizing the Potential of Teacher Residencies.” Bellwether 
Education Partners (July 18, 2018). 


A See Rockman et al, “Measures of Success” (2015) and “The MASTER Model: Preparation through Partnerships” (2017). Because we 
lacked longer-term data from MASTER, no figures are included in the Achievement study; the Retention study does include MASTER 


figures. 


: John P. Papay, Martin R. West, Jon B. Fullerton, and Thomas Kane, “Does an Urban Teacher Residency Increase Student 


Achievement? Early Evidence from Boston,” Educational Evaluation and Policy Analysis (2012). See also Linda Darling-Hammond, et al., 
“Does Teacher Preparation Matter? Evidence about Teacher Certification, Teach for America, and Teacher Effectiveness,” Education 
Policy Analysis Archives, [S.I.], v. 13, (Oct. 2005): 42. Available at: 

<https://epaa.asu.edu/ojs/article/view/1 47>. 


program completion and licensure requirements, that meet the need for “completer- and program-level data” 
encouraged in the Bellwether report to “measure the effects of their own improvement efforts.”* The key may be that, 
providing ongoing feedback to residents and mentors, UTR partners hold the model and its participants to account 
during their residency year—successtully accelerating the learning curve of UTR graduates. 


The Special Education study perhaps best tells the counter narrative proposed in the Bellwether report, providing 
evidence of “practices and processes” that lead to positive outcomes.” Like all UTR teachers, UTR special educators are 


trained in formative assessment and the use of data to diagnose and address students’ learning needs. But for novice 
special educators, the demands of addressing multiple learning needs, teaching unfamiliar content, and writing detailed 
Individual Education Plans (IEPs) may exceed their training and experience. 


These challenges are often the focus of UTR partners’ continuous improvement efforts, but our case study findngs 
suggest that the key skill is knowing when to pivot if a strategy isn’t working. This flexibility allows UTR special educators 
to try a different tack or put their heads together with co-teachers to devise new strategies. Ultimately, it helps them 
acclimate to urban schools and contribute to them—both of which contribute to job satisfaction and thus retention. 
Retention data seem to bear this out: UTR special educators tend not just to stay but to stay put: 73% are still teaching 
where they started. This can bring stability to schools, and may account for the fact that 11th and 12th grade special 
education students in schools with UTR-trained special educators had statistically significant higher attendance rates and 
earned more credits than special education students in schools with no UTR-trained teachers. This constellation of links 
between training, practice, and outcomes may be what makes UTR a viable, durable, and in many ways different model 
for teacher preparation. 


What also sets UTR apart is that it was always about more than preparing residents. Its legacy is its influence on mentors 
as well, on the principals and teachers in host and hiring schools, on the partners who continue to revise and adapt the 
UTR model—and on their many colleagues deeply committed to preparing teachers and improving schools. 


4 | Betti and Trinidad, 21, 25. 


© | aBetti and Trinidad, 21. 


Chapter 1 
Training Teachers to 
Increase Student Performance 


Our achievement study builds on research and focuses on the impact of UTR graduates, comparing the performance of 
their secondary students to that of students taught by other teachers with the same level of classroom experience. The 
study poses two guiding questions: 


How do UTR-trained teachers compare to non-UTR 


, Does their impact change over time, as years 
teachers, based on their students’ performance on the is g y 


New York State Regents exams? 


of teaching experience increase? 


The first question focuses on how UTR-trained teachers compare to other teachers in terms of their students’ Regents 
exam performance. The second question explores whether the performance of students of UTR-trained teachers 
improves as teachers gain experience. 


Organization of the Chapter 


This chapter is organized into two sections, the first of which provides an analysis of how UTR-trained teachers’ 
students’ performance on subject-specific Regents tests compares to that of students taught by non-UTR-trained 
teachers with the same number of years of experience, teaching the same subject.® The second section examines 
interaction effects more closely, looking at whether student performance changes as UTR graduates gain experience, 
and whether similar links emerge between special education students’ performance and content area teachers’ 
experience. 


METHODS 


This study primarily relies on previously collected achievement data for UTR Cohorts 1-7, or Regents exam scores from 


2010-11 through 2016-17, as applicable to each cohort. Previous analyses included UTR residents’ students, whose 


° Teachers in both the UTR and non-UTR samples were drawn from the 70+ schools in the New Visions network, one of the largest 
affinity groups supporting NYC DOE schools. Drawing the study sample from network schools expedited data access and ensured that 
in-school support was similar across teachers and thus not a confounding factor. 


performance was compared to that of students taught by other first-year teachers; this study excluded residents, 
focusing only on graduates once they become teachers of record. 


Previous analyses also compared UTR- versus non-UTR trained teachers within the New Visions network, but within a 
single year of achievement data and a single cohort. This study builds on that work by combining data across multiple 
years and examining the relative advantage of being a UTR-trained teacher across various time points (e.g., during the 
first year of teaching, during the second year, etc.). With this design, examining the “UTR effect” during their first year of 
teaching, analyses could potentially include teachers from all seven UTR cohorts and from all years of achievement data, 
thereby maximizing sample size and allowing us to synthesize findings across waves of data collection. 


Unfortunately, relying on previously collected data has some drawbacks: any flaws in the earlier waves of data 
collection—unbalanced samples for Regents scores, teachers without matches, missing data, changes in the variables 
included in each file over the years—could continue to have an impact. While the general matching approach was the 
same across all waves of data collection, the extent to which the matching was successful and/or documented (in terms 
of specific IDs for matching “pairs” of teachers) varied. Each year, we used Mahalanobis Distance (MD) scores, based on 
school demographics (% economically disadvantaged, % Hispanic, % African American, % students with disabilities, % 
English language learners, % female, and total enrollment), to identify similar schools. We then matched UTR teachers 
within each school to other teachers in schools with similar MD scores, who had the same number of years of teaching 
experience and were teaching similar courses (excluding Advanced Placement, remedial, or Credit Recovery courses). 
We then examined Regents scores for the students in these courses to determine if there were significant differences 
between the performance of students of UTR-trained teachers versus non-UTR trained teachers. We compared scores 
using analysis of covariance (ANCOVA), controlling for gender, English language learner and special education status, 
status, ethnicity (Hispanic and African American), and prior achievement (8th grade reading and math scores).’ 

Overall, data from 71 UTR teachers were included in the study. Of these, 29 taught courses that led to English Regents 
exams; 19, courses that led to one or more math Regents exams (Algebra |, Geometry, or Algebra Il-Trig); and 26, 
courses that led to science Regents exams (living environment, chemistry, or earth science). (Three teachers taught 
multiple subjects, which is why the counts add up to more than 71). The representation of UTR cohorts across the 
different analysis files is summarized in Table 1 below; Table 2 shows the number of students by subject and UTR 
teachers’ year in the classroom. (The numbers of students in the comparison group varies, and are included in Tables 3- 
7.) An important caveat of our “longitudinal” analyses that becomes apparent in the table is that it is impossible to 
separate the effect of cohort from years of teaching experience —earlier cohorts have had time to accumulate more years 
of experience and therefore are the ones represented in those “experienced” files. Due to sample size constraints, the 
fifth through seventh years of teaching experience files were combined for analysis. We also created an overall file, 
combining all of the individual teaching year files, for a total of 166 UTR teacher/year instances (along with their non-UTR 
matches). 

Not included in this study are teachers trained in the Mathematics and Science Teacher Education Residency (MASTER) 
program, a three-year, NSF-funded partnership between Hunter, New Visions, and the NYC DOE that overlapped with 


. Our original analysis plan for this longitudinal study used HLM to model the effects of UTR over time. However, because we only have 
“within teacher” data over time for UTR teachers and not the matched sample (matches tended to be different each year), we were 
unable to construct a model where Level 1 represented the longitudinal data for each teacher in the full sample. Our revised plan was to 
use HLM to exam the UTR effect within a single year of teaching experience (e.g., data from only a teacher’s 2nd year or teaching), 
unfortunately we found that our sample sizes for Level 2 (teacher level) within each subject area were not large enough in most 
instances. We thus determined that the best approach was to use Analysis of Covariance (ANCOVA) as we had in previous waves of 
analysis. 


UTR and also employed the residency model. 


Because the MASTER project ended in 2016, it did not allow us to look at 


longitudinal effects. Excluding MASTER did reduce the number of math and science teachers in certain years, since 


those subjects were covered by MASTER rather than UTR, but the overlap complicated isolating a project effect. Our 


analyses do include two Cohort 7 English teachers participating in the Learning Partners Urban Teacher Residency (LP- 


UTR), a continuation of the residency model that added features of the NYC DOE’s Learning Partners program. These 


two teachers are included only in analyses for teachers in their first year and in analyses for overall years. 


Table 1. Achievement Analysis Sample—Number of Teachers by UTR Cohort 


Number of Teachers 


Cohort 1 Cohort2  Cohort3  Cohort4 CohortS5 Cohort6 Cohort7 All Cohorts 
1st Year of Teaching 5 14 4 8 8 4 2 45 
2nd Year of Teaching 6 8 7 5 7 1 6) 34 
8rd Year of Teaching 3 13 6 6 7 0 0 35 
4th Year of Teaching 3 11 3 7 0 0 0 24 
5th Year of Teaching 2 6 0 0 0 0 16 
6th Year of Teaching 3 ) 0 ) 0 ) 14 
7th Year of Teaching 1 0 0 0 0 0 1 
Total # of 
Teacher/Year 23 62 26 26 22 5 2 166 
Instances 
ee 9 21 11 12 12 4 2 71 


Teachers 


Table 2. Achievement Analysis Sample—Number of Students by UTR Teachers’ Year in the 
Classroom 


Number of Students 


5th-7th Year 4th Year 3rd Year 2nd Year 1st Year 
Integrated Algebra 0 159 108 171 235 
Geometry 0 0 175 126 66 
Alg 2 Trig 64 0 0 0 60 
Living Environment 351 306 360 406 509 
Chemistry 0 236 210 260 362 
Earth Science 0 120 0 175 168 
English 327 299 503 491 611 


Sample sizes within each set of analyses (representing a particular year of teaching experience and Regents exam) varied 


greatly by year and subject area and are specified within the outcome tables in the results section. Only exams where 


data were available for at least two UTR and two comparison teachers with at least 30 students each were analyzed. 


There are other subject-related factors that may affect student performance and thus findings, including New York’s 
2010 adoption of the Common Core Standards for Mathematics, and, during the two-four years afterward, the 
staggered shifts to the Common Core aligned state Regents exams for Algebra |, Geometry, and Algebra Il-Trig. The 
math scores used in our analyses for the years prior to the changes refer to the old exams. There may have been a few 
cases where students took the old and new exams, but schools entered only one score, likely the higher one. In cases 
where the goal was to examine performance across years by subject, we combined scores. 


Another factor that we considered in our analyses is the timing of exams: for example, students sometimes sit for the 
Regents exam in English in their junior year. For the study, we used scores only of Regents that were taken during the 
year of the "file,” so if we happen to have 10th grade students with English Regents, it is because they took it that year. 
In the rare cases where multiple attempts were made during the same year, we used the highest score for that exam. 


SUMMARY OF FINDINGS 


CHERS' EXPERIENCE AND STUDENT ACHIEVEMENT 


Regents exam scores across all subject areas showed that UTR training resulted in either a 
significant advantage or “no harm done.” Earlier findings indicated that UTR had a positive impact on 
student achievement, with students performing as well as or better than peers taught by other early career 
teachers. The results shared here confirm the earlier—positive— findings. 


In comparisons where differences in students’ Regents scores were statistically significant, the UTR 
group’s performance was higher 89% of the time. Overall, we conducted 27 comparisons, examining 
adjusted means of Regents scores of students taught by UTR- and non-UTR-trained teachers with the same 
number of years of classroom experience. Differences in student performance were not dramatic, but students 
taught by UTR-trained teachers outperformed peers in eight out of nine comparisons where the differences in 
scores were statistically significant. 


There was only one instance where the performance of UTR teachers’ students was significantly 
lower than that of students taught by non-UTR teachers with the same number of years of 
experience. For teachers in their third year of teaching, students taught by the non-UTR group outperformed 
the UTR group on the Integrated Algebra Regents exam. 


Students taught by UTR-trained teachers with varied years of experience performed significantly 
higher across multiple subject-specific Regents exams compared to the performance of students of non- 
UTR teachers with the same number of years of teaching experience. 


Students of UTR teachers performed significantly higher on the Regents geometry, living 
environment, earth science, and English exams, based on comparisons of the performance of students of 


UTR teachers across all available years of teaching experience, 


IN] 


ERACTION EFFECTS 


In some math and science subjects, the UTR benefit may become stronger over time. For geometry, 
algebra 2 — trigonometry, and earth science, our longitudinal analyses of the interaction between the UTR effect 
and years of teaching experience, showed that, in some instances, UTR teachers may become more effective 
as they gain classroom experience: a positive gap between UTR vs. non-UTR was more evident in the group 
with three or more years or teaching experience in all three subjects. For chemistry, the analysis suggests that 
the initial UTR benefit may disappear over time. 


Whether or not a UTR teacher is a career changer may affect student performance. Our comparisons 


within the UTR group showed a statistically significant difference in the performance of students taught by UTR 
teachers who were “career changers, (those who began the UTR program six or more years after completing 


10 


their undergraduate degree, and the performance of students taught by “non-career changers,” those who 
began UTR within five years. Students in the career changers group performed significantly lower on the 
Regents English exam than those who were taught by the “non-career changers.” 


Level of education may also factor into teachers’ impact on student performance. Again looking within 
the UTR group, our analysis found a statistically significant difference in the performance of students taught by 
UTR teachers who began the program with only an undergraduate degree, compared to UTR teachers with a 
Master’s or Doctorate. Students in the “advanced degrees” group performed significantly lower on the Regents 
English exam and on the Regents Living Environment exam. 


Results for the impact of UTR-trained teachers in Integrated Co-Teaching (ICT) classrooms were 
mixed. Earlier analyses had suggested that UTR training may help narrow the achievement gap between 
special and general education students. The analyses conducted for this study did not show this effect, though 
the analyses did indicate a positive effect for general education students. In examining the interaction between 
Special Education and UTR, we found significant interaction for Living Environment in which the benefit of being 
taught by a UTR-trained content teacher working in an ICT classroom was evident only for students who did 
not have an IEP. Findings between Special Education and UTR across all other subject areas were not 
significant. (For additional insight on the impact of UTR trained special educators please see the Special 
Education Report, the second chapter in this series of supplemental studies.) 


1 


section 1 
Student Achievement by 
Years of Teaching 


For each UTR cohort, from 2009—2015, we compared the performance of students taught by UTR-trained teachers to 
that of students taught by teachers matched by years of experience and course, gradually adding analyses of not only 
UTR residents’ impact but also graduates’. And, each year, we found that students taught by UTR-trained teachers were 
doing as well as or better than their peers. Where differences between the two groups were statistically significant, 
results favored UTR. 


Analyses conducted as part of our 2016 summative study, which examined UTR graduates’ impact one, two, three, and 
four years beyond their residency, again indicated that UTR provided a significant advantage, or graduated teachers 
whose students performed on par with their peers. Adding data from three more years, the supplemental study allowed 
us to lengthen the time frame, merge annual findings, and broaden our view. While clear trends or results pointing to a 
definitive sweet spot remain elusive, the findings shared below confirm earlier findings, with UTR still holding its edge: 
About a third, or nine of the 26 comparisons we conducted showed statistically significant differences in the Regents 
scores of students taught by UTR-trained graduates and those taught by other early career teachers: in eight out of nine 
or 89% of those comparisons, the UTR group outperformed peers. 


TEACHING EXPERIENCE AND STUDENT ACHIEVEMENT 


1st Year of Teaching 


Looking at the performance of students of UTR teachers during their first year of teaching and comparing it to the 
performance of students of non-UTR teachers during their first year of teaching, we found two statistically significant 
differences across the Regents exams.® 


Students in the UTR group performed significantly higher on the Regents living environment exam (UTR adjusted 
mean=71.1, Non-UTR adjusted mean=65.3, F(1,910)=62.2, p<.001) and significantly higher on the Regents chemistry 
exam (UTR adjusted mean=66.1, Non-UTR adjusted mean=63.6, F(1,544)=6.2, p=.01). These analyses (ANCOVA) 
controlled for gender, special education status, English language leamer status, ethnicity (Black or Hispanic) and sth 
grade proficiency scores in reading and math. 


8 The shading in Tables 2-7 below highlight statistically significant differences, with p values at .05 or higher, between the UTR and non- 
UTR groups: green indicates the higher score or adjusted mean. 
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Table 3. 1st Year of Teaching 


UTR Non-UTR 
Regents N N Adjusted SE N N Adjusted SE Sig of 
Exam Teachers Students Mean Teachers Students Mean Difference 
Inipgtalod 6 235 63.0 07 5 180 61.2 08 |  p=10 
Algebra 
Geometry 5 56 55.8 1.7 5 124 59.6 led p=.07 
Alg 2 Trig 4 60 58.3 2.1 4 94 63.1 1.6 p=.11 
Ewing 10 509 71.4 0.5 11 418 65.3 05) p<.00t 
Environment 
Chemistry 4 362 66.1 0.6 4 191 63.6 0.8 p=.01 
Este 2 168 60.2 1.0 2 70 59.1 1.8 p=.63 
cience 
English 18 611 ralerg 0.5 19 501 71.8 0.6 p=.89 


2nd Year of Teaching 


Comparisons of UTR teachers’ students’ performance to that of students of non-UTR teachers, during teachers’ second 
year, showed only one statistically significant difference—close to 10 points—across the Regents exams. Students in 
the UTR group performed significantly higher on the Regents earth science exam (UTR adjusted mean=69.0, Non-UTR 
adjusted mean=59.7, F(1,294)=33.7, p<.001). Analyses (ANCOVA) again controlled for gender, special education status, 
English language learner status, ethnicity (Black or Hispanic) and 8th grade reading and math proficiency. 


Table 4. 2nd Year of Teaching 


UTR Non-UTR 
Regents N N Adjusted SE N N Adjusted SE Sig of 
Exam Teachers Students Mean Teachers Students Mean Difference 
ume ieee 5 171 63.7 0.7 6 ary 64.2 0.6 p=.62 
Algebra 
Geometry 3 126 67.0 1.2 2 135 68.3 Tal p=.45 
ng ? 406 69.4 0.5 8 421 69.1 0.5 p=.73 
Environment 
Chemistry 4 260 64.1 0.7 4 82 64.2 1.3 p=.96 
Se itln 2 175 69.0 1.0 3 128 597 121| p<.001 
Science 
English 13 491 68.7 0.6 13 353 66.9 0.7 p=.10 


3rd Year of Teaching 


Results were similar after the third year of teaching, with no discernible trends. Comparisons of the performance of UTR- 
taught students during teachers’ third year of teaching to the performance of students of non-UTR teachers, also during 
their third year, showed three statistically significant differences across the Regents exams—two with more favorable 


13 


outcomes for the UTR group and one with more favorable outcomes for the non-UTR group. This was the only outcome 
favoring the non-UTR group. 


Students in the UTR group performed significantly higher on the Regents geometry exam (UTR adjusted mean=68.0, 
Non-UTR adjusted mean=57.3, F(1,320)=52.0, p<.001) and on the Regents living environment exam (UTR adjusted 
mean=73.6, Non-UTR adjusted mean=70.0, F(1,689)=14.9, p<.001). UTR students performed significantly lower than 
the non-UTR students on the integrated algebra exam (UTR adjusted mean=61.6, Non-UTR adjusted mean=64.2, 
F(1,185)=4.0, p=.048). These analyses (ANCOVA) controlled for gender, special education status, English language 
learner status, ethnicity (Black or Hispanic) and 8th grade proficiency scores in reading and math. 


Table 5. 38rd Year of Teaching 


UTR Non-UTR 
Regents N N Adjusted SE N N Adjusted SE Sig of 
Exam Teachers Students Mean Teachers Students Mean Difference 
Integrated | _ 
Algebra _| 3 108 61.6 0.8 3 86 64.2 0.9 p=.048 
Geometry 5 175 68.0 1.0 4 154 57.3 1.0 p<.001 
Living 
Environment 9 360 73.6 0.6 8 340 70.0 0.6 p<.001 
Chemistry 5 210 60.4 0.7 4 68 61.6 1.3 p=.39 
English 15 503 69.9 0.7 14 408 68.5 0.7 p=.14 


4th Year of Teaching 


Fourth-year comparisons again showed fairly comparable performances, with two notable exceptions. Specifically, 
students in the UTR group performed significantly higher on the Regents earth science exam (UTR adjusted mean=69.8, 
Non-UTR adjusted mean=56.2, F(1,156)=31.4, p<.001). This was the widest margin in all out comparisons, with a 
difference of 13.6 points. 


On the Regents English exam (UTR adjusted mean=75.6, Non-UTR adjusted mean=72.9, F(1,493)=4.6, p=.03), the UTR 
group also outperformed peers—the only instance among all the comparisons of English Regents performance where 
differences between the two groups were statistically significant. Again, these analyses (ANCOVA) controlled for gender, 
special education status, English language learner status, ethnicity (Black or Hispanic) and 8th grade proficiency scores 
in reading and math. 
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Table 6. 4th Year of Teaching 


UTR Non-UTR 
Regents N N Adjusted SE N N Adjusted SE Sig of 
Exam Teachers Students Mean Teachers Students Mean Difference 
nei 4 159 63.0 0.8 3 110 62.5 1.0 p=.69 
Algebra 
ening 7 306 73.1 0.6 6 146 72.2 0.9 p=.44 
Environment 
Chemistry 5 236 60.3 0.8 4 108 61.5 1.2 p=.41 
Earth 
Garenias 2 120 69.8 1.2 2 46 56.2 2.0 p<.001 
English 8 299 75.6 0.8 v4 203 72.9 1.0 p=.03 
5th - 7th Year of Teaching 
As mentioned earlier, our sample sizes for the sixth and seventh year of teaching were very small, so those data were 
combined with the data representing the fifth year of teaching, for a combined file representing the fifth through the 
seventh year of teaching. 
The combined comparisons revealed only one statistically significant difference. Specifically, students in the UTR group 
performed significantly higher on the Regents algebra 2 — trigonometry exam (UTR adjusted mean=64.0, Non-UTR 
adjusted mean=53.8, F(1,132)=20.8, p<.001). These analyses (ANCOVA) controlled for the same factors as earlier ones. 
Table 7. 5th - 7th Year of Teaching 
UTR Non-UTR 
Regents N N Adjusted SE N N Adjusted SE Sig of 
Exam Teachers Students Mean Teachers Students Mean Difference 
Alg 2 Trig 2 64 64.0 1.6 2 77 53.8 1.4 p<.001 
une 9 351 71.2 0.6 9 452 70.0 05 p<.16 
Environment 
English 13 327 72.1 0.8 14 336 72.8 0.8 p=.50 


FINDINGS BY ALL YEARS COMBINED 


n order to facilitate a synthesis of findings across all waves of data collection, representing all cohorts across multiple 


years of teaching, we created a combined file from the first through seventh year of teaching files. As seen across the 
single year results (Tables 3-7), the areas in which UTR students significantly outperformed their non-UTR peers varied. 
For example, for earth science, although the trend showed that the UTR group had a higher adjusted mean whenever 
measured, that advantage only reached statistical significance in the second and fourth years of teaching. During the first 
year, it was not statistically significant, and, in the third and first through seventh-year files, there weren’t enough 
teachers/students to analyze the findings within that subject area. 


Table 8 shows that when all files were combined, there was a clear overall advantage of being taught by a UTR-trained 
teacher compared to a non-UTR trained teacher. These analyses controlled for all of the previous covariates (gender, 
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special education status, English language learner status, ethnicity, and 8th grade proficiency scores in reading and 
math), as well as the additional covariate of years of teaching experience.° 


Comparing the performance of students of UTR teachers across all available years of teaching experience to the 
performance of students of non-UTR teachers with matched years of teaching experience, we found that UTR training 
resulted in either a significant advantage or “no harm done” in terms of Regents exam scores across all subject areas. 
There were no instances in which the students of non-UTR teachers significantly outperformed the students of UTR 
teachers for this overall “all years combined” sample. 


Specifically, students in the UTR group performed significantly higher on the Regents geometry exam (UTR adjusted 
mean=65.8, Non-UTR adjusted mean=59.9, F(1,932)=42.0, p<.001), on the Regents living environment exam (UTR 
adjusted mean=71.7, Non-UTR adjusted mean=68.7, F(1,3685)=64.0, p<.001), on the Regents earth science exam 
(UTR adjusted mean=67.5, Non-UTR adjusted mean=59.5, F(1,929)=78.4, p<.001), and on the Regents English exam 
(UTR adjusted mean=71.3, Non-UTR adjusted mean=70.3, F(1,4015)=4.6, p=.08). 


Table 8. All Years Combined 


UTR Non-UTR 
N N 
Regents Teachers N Adjusted SE Teachers N Adjusted SE Sig of 
Exam (unique Students Mean (unique Students Mean Difference 
teachers) teachers) 
Integrated = 
Algebra 20 (13) 702 62.9 0.4 19 (19) 677 62.7 0.4 p=.66 
Geometry 16 (10) 439 65.8 0.6 15 (13) 503 59.9 0.6 p<.001 
Alg 2 Trig 9 (5) 227 58.9 1.0 9 (8) 244 59.1 1.0 p=.90 
Ewing 42 (20) 1932 71.7 03] 42(34) 1777 68.7 03 001 
Environment ; : é , pe 
Chemistry 19 (7) 1164 62.9 0.3 17 (13) 469 62.5 0.5 p=.49 
pace 8 (3) 607 67.5 0.5 9 9) 333 59.5 0.7 p<.001 
cience 
English 67 (29) 2231 71.3 0.3 67 (53) 1801 70.3 0.3 p=.03 


' In this file, a teacher could be represented multiple times (during different years of teaching experience) and each combination is 
counted as a separate sample case for this analysis. The “unique” teacher count is represented in parentheses. In general, UTR 
teachers were matched with different non-UTR teachers each wave. For example, in earth science, there were three unique teachers: 
two were represented in two different years and one was represented in four different years for a total of eight teacher/year cases. Each 
year the “match” was different, and in one situation two separate teachers were used as matches in order to increase the student 
sample size for the non-UTR sample (resulting in a final non-UTR teacher sample of nine). 
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section ? 
Interaction Effects 


As part of our summative evaluation, we looked for trends by subject area, finding that for certain subjects and Regents 
performances, UTR training appeared to have a more pronounced effect. Students taught by UTR-trained teachers, for 
example, seemed to be outperforming their peers on the Living Environment Regents exams with some consistency. 
Throughout UTR, as part of our efforts to detect any impacts on special education, not always apparent in standardized 
achievement measures, we also looked for indications that UTR training was helping to close the achievement gaps 
between special and general education students. 


For the supplemental study, we again looked at differences by subject, but more specifically at the interactions between 
subject and years of teaching. What we found in some cases confirmed our earlier findings, but, perhaps more 
important, pointed to where initial UTR benefits seemed to disappear—or strengthen—over time. We also examined 
interactions based on prior experience and schooling, comparing the performance of students taught by UTR-trained 
teachers who were career changers to that of students taught by non-career changers, and the performance of students 
taught by UTR teachers who began the program with an undergraduate degree only versus those who held a Masters or 
Doctorate prior to the residency. We again explored UTR’s impact on special education, through the lens of interactions. 


DIFFERENCES OVER TIME AND BY SUBJECI 


Years of Teaching Experience 


We examined the interaction between the UTR effect and years of teaching experience, comparing those taught by 
teachers with one or two years of experience to those taught by teachers with three or more years. (See Figures 2 
through 7 below.) We found significant interactions for geometry (F(1,931)=62.6, p<.001), algebra 2 - trig (F(1,460)=8.4, 
p=.01), chemistry (F(1,1620)=6.2, p=.01), and earth science (F(1,928)=9.9, p=.01). The results suggest that for geometry, 
algebra 2 - trigonometry and earth science, the UTR benefit may become stronger over time (a positive gap between 
UTR vs. non-UTR is more evident in three years or more group). The pattern for chemistry suggests that the initial benefit 
may disappear over time. The remaining subject areas did not have significant interaction effects but are still included for 
reference in the charts on the following pages. 
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Figure 1. Integrated Algebra: Interaction 
between Years Exp x UTR (not sig) 
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Figure 3. Algebra 2 Trig: Interaction 
between Years Exp x UTR 
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Figure 5. Chemistry: Interaction between 
Years Exp x UTR 
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Figure 2. Geometry: Interaction 
between Years Exp x UTR 
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Figure 4. Living Environment: 


Interaction between Years Exp x UTR 
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Figure 6. Earth Science: Interaction 
between Years Exp x UTR 
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Figure 7. English: Interaction between 
Years Exp x UTR (not sig) 
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Career Changers vs. Recent Grads 


Looking just within the UTR group, we compared the performance of students taught by UTR teachers who were “career 
changers,” or those who began the UTR program six or more years after completing their undergraduate degree, to the 
performance of students taught by “non-career changers,” those who began UTR within five years. Our analyses were 
limited to data for the first year of teaching due to sample size constraints in later years. We found one statistically 
significant difference across the Regents exams with sufficient sample size. Specifically, students in the “career changer” 
group performed significantly lower on the Regents English exam than those who were taught by the “non-career 
changers” (career changer adjusted mean=70.8, non-career changer adjusted mean=75.8, F(1,600)=16.7, p<.001). 
These analyses (ANCOVA) controlled for gender, special education status, English language learner status, ethnicity 
(Black or Hispanic) and 8th grade proficiency scores in reading and math. 


Table 9. UTR Teacher Performance by Prior Career Status 


Career Changer Non-Career Changers (Recent 
Graduates) 
Regents N N Adjusted SE N N Adjusted SE Sig of 
Exam Teachers Students Mean Teachers Students Mean Difference 
English 7 189 70.8 0.97 1 420 75.8 0.60 p<.001 
imeateled 3 63 62.9 1.31 2 117 645 0.91 p=.35 
Algebra 
Ewing 3 166 72.3 0.84 6 334 70.4 0.58 p=.06 
Environment 


Undergraduate Degree Only vs. Masters or Doctorate 


Looking just within the UTR group, we compared the performance of students taught by UTR teachers who had 
previously earned advanced degrees (masters or doctorate) versus a bachelor’s degree only prior to beginning the UTR 
program. Again, our analyses were limited to data for the first year of teaching due to sample size constraints in later 
years. We found that in both subject areas with sufficient sample size, students taught by those with advanced degrees 
had lower scores. Specifically, students in the “advanced degrees” group performed significantly lower on the Regents 
English exam (advanced degree adjusted mean=72.8, bachelor’s degree adjusted mean=75.3, F(1,560)=5.5, p=.02) and 
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on the Regents Living Environment exam (advanced degree adjusted mean=66.6, bachelor’s degree adjusted 
mean=72.1, F(1,491)=17.7, p<.001). These analyses (ANCOVA) controlled for gender, special education status, English 
language learner status, ethnicity (Black or Hispanic) and 8th grade proficiency scores in reading and math. 


Table 10. UTR Teacher Performance by Educational Attainment 


Advanced Degree Bachelor’s Degree Only 
Regents N N Adjusted SE N N Adjusted SE Sig of 
Exam Teachers Students Mean Teachers Students Mean Difference 
English 6 279 72.8 0.73 11 290 75.3 0.71 p=.02 
Ewing 2 100 66.6 1.13 7 400 72.4 0.52 | p<.001 
Environment 


Important Caveats Regarding Longitudinal Effects 


While one of the primary research objectives was to examine the UTR effect over time, it is critical to note that we do not 


have the ideal sample from a statistical perspective. Ideally, the sample would consist of a set of UTR teachers and a 


matched set of non-UTR teachers, all of whom would be followed for several years with valid test data each and every 
year in the same subject area. That is not the type of data that UTR had available. Teachers typically had data in only two 


or three waves of data collection and the “match” differed by year (i.e., matches typically did not have longitudinal data). 


Also, aS mentioned previously, earlier cohorts are over-represented in later years of teaching experience data, which 


confounds our results (e.g., if earlier cohorts had a less effective program experience, that could disproportionately affect 


the results seen for the 5th-7th year of teaching experience analyses). That said, as a way of graphically 


representing/summarizing the scores across the teaching years, Figures 8 through 14 below depict the adjusted mean 


scores from Tables 2-6, organized by subject area. Please note that not all subject areas had valid comparisons across 


all years, samples vary over time, and not all differences seen are statistically significant. 
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Figure 8. Integrated Algebra: UTR vs. Non-UTR Teachers over Time 
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Figure 9. Geometry: UTR vs. Non-UTR Teachers over Time 
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Figure 10. Algebra 2 / Trig: UTR vs. Non-UTR Teachers over Time 
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Figure 11. Living Environment: UTR vs. Non-UTR Teachers over Time 
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Figure 12. Chemistry: UTR vs. Non-UTR Teachers over Time 
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Figure 13. Earth Science: UTR vs. Non-UTR Teachers over Time 
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Figure 14. English: UTR vs. Non-UTR Teachers over Time 
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RAINING AND SPECIAL EDUCATION 


INTERACTIONS BETWEEN 


Combined Special Education Files 


As noted above, for each UTR cohort, we examined the interactions between UTR training and special education to 
explore whether having a UTR-trained teacher helped narrow the gap between regular and special education students. 
We reported some positive effects each year, but findings were not consistent by subject area nor conclusive. 


For this study, within the combined file, we also examined whether or not there was a significant interaction between 
Special Education and UTR. We found significant interaction for Living Environment in which the benefit of being taught 
by a UTR teacher was evident only for students who did not have an IEP. As illustrated in Figure 15 below, for students 
with an IEP, their exam scores were more similar to their peers (also with IEPs) taught by non-UTR teachers (F 
(1,8684)=5.0, p=.03). No significant interactions were found for the other subject areas. 


Figure 15. Living Environment: Interaction between Special 
Education x UTR Status 
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Conclusions and 
Recommendations 


Our analyses indicate that there is an overall advantage of being taught by a UTR-trained teacher compared to a 
non-UTR trained teacher. Comparing the performance of students of UTR teachers across all available years of 
teaching experience to the performance of students of non-UTR teachers with matched years of teaching 
experience, we found that UTR training resulted in either a significant advantage or “no harm done,” based on 
Regents exam scores across all subject areas. These analyses controlled for covariates including gender, special 
education status, English language learner status, ethnicity (Black or Hispanic) and 8th grade proficiency scores in 
reading and math, as well as the additional covariate of years of teaching experience. 


Looking at the interaction effects within the UTR group suggests that for some subject areas the benefits of the 
residency program are strengthened the longer residents are in the classroom (geometry, algebra 2-trig, and earth 
science while for another subject area (chemistry) the benefit may fade over time. Residents’ prior career 
experience and educational attainment also significantly influence the achievement of their students when 
compared to their UTR peers. Our analysis indicated that more career experience and higher degrees may not lead 
to higher student achievement. This finding may have implications for recruiting for residencies. 


Our recommendations focus on some additional analyses to consider as UTR program graduates gain more 
experience in the classroom, in tandem with some programmatic changes. 


« Identify elements of the UTR residency that vary or are subject-dependent, perhaps isolating these 
differences to understand more about initial impacts, impacts that build or fade over time—and ways 
residency training and support might deepen, accelerate, or mitigate differences. 


= — Review the performance of career changers and residents who enter the program with a graduate degree 
to identify ways the residency program could be modified or training differentiated to leverage 
background and skills but still close a possible gap in how their students perform. 


=» Explore ways to assess the impact of UTR trained special educators on the performance of students in 
the ICT classrooms in which they serve. 


"Continue to explore ways that UTR-trained educators, over time, have an impact not measured by test 
scores. These could include examining other institutional measures such as attendance or progress 
toward graduation, or gauging contributions to school communities, including participating on leadership 
or curriculum teams, or taking part in teacher-led staff development. 
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Chapter 2 
Educating All Students 


UTR’s Educational Environment 


The chapter’s title comes from the New York State credentialing exam, Educating All Students (EAS), which 
requires all teachers licensed by the state to demonstrate that they are prepared to teach all students. EAS was 
created in response to an increasingly diverse student population—English learners, students newly eligible for 
services due to expanded definitions and awareness of special needs— and changes in state and city-wide special 
education policies. The most notable changes, piloted in 2009-2010 by the New York City Department of 
Education (NYCDOE) and launched city-wide in 2012 as part of A Shared Path to Success reforms, required a shift 
to Integrated Co-Teaching (ICT).'' In the ICT model, students with disabilities are not consigned to separate 
classrooms, but learn alongside non-disabled peers in classrooms where content and special education teachers 
share responsibilities. The model, and the EAS exam, in many ways ask all teachers to do what has long been the 
charge to special educators: to meet the needs of all students. 


If inclusion policies asked general education teachers to do what special educators do as a matter of course, other 
legislation did the reverse. The 2004 reauthorization of the Individuals with Disabilities Education Improvement Act 
(IDEA), the 1975 landmark legislation that defined disabilities and required public schools to provide all eligible 
students with “free and appropriate public education,” brought /DEA into line with legislative mandates set by No 
Child Left Behind (NCLB) and the Elementary and Secondary Education Act (ESEA): it mandated that special 
education teachers, like all public school elementary and secondary teachers, be “highly qualified.” ' 


Efforts to put effective teachers in front of all children, mandated by IDEA and bolstered by research showing that 
teacher quality is the best school predictor of students’ success, '® also lent support to other school reform efforts. 
There was, for example, new federal funding to improve teacher training—including the US Department of 
Education’s Teacher Quality Partnership program—along with large-scale research efforts designed to define 
quality and assess practice, such as the Gates Foundation’s Measuring Effective Teaching (MET) project. The focus 
on quality also led states and local agencies to introduce ways to evaluate teachers, many patterned after the 
Danielson Framework for Teaching, adopted by the NYCDOE, and incorporated in UTR’s third year as part of its 


resident assessment suite. 


" Fora history and summary of the city’s efforts, see NYC DOE’s “Family Guide to Special Education Services for School-Age 
Children: A Shared Path to Success,” available at http://schools.nyc.gov/NR/rdonlyres/DBD4EB3A-6D3B-496D-8CB2- 
C742F9B9AB5C/0/Parent_Guide_for_Students_with_Disabilites_Updated_Web.pdf. 

% For a review of IDEA, ESEA, and NCLB legislation, see http://idea-b.ed.govIDEA] [20 U.S.C. 1401(10). See also 
www.understood.org/en/school-learning/your-childs-rights/basics-about-childs-rights/at-a-glance-free-and-appropriate-public- 
education. 

13 See Annie Lowery, “Big Study Links Good Teachers to Lasting Gain,” New York Times, Jan. 6, 2012. See also Raj Chetty, 
John N. Friedman, and Jonah E. Rocko, “The Long-Term Impacts of Teachers.” Working Paper 17699 (National Bureau of 
Economic Research, 2011), available at http://www.rajchetty.com/chettyles/value_added.pdf. 
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Though improving teacher quality would seem an unassailable goal, efforts were not without controversy. Some 
were related to ongoing controversies sparked by NCLB, some to new accountability metrics that teachers felt 
were punitive, and some to a belief that school staffing challenges were a matter not of quality, but of persistent 
inequality.'* For some time, researchers and policymakers had called for more equitable distribution of teacher 
talent as a way to address teacher shortages and turnover in low-performing, under-resourced schools, many of 
them in urban areas. It was this acute need that had led to Teach for America, AmeriCorps programs, and other 
efforts to re-think and re-build the teaching corps. Schools of education also sought ways to attract talent to the 
profession by adding alternative paths to certification programs to their traditional programs. 


Growing needs, a reform environment, funding opportunities, and a long-standing partnership led the country’s 
largest school district, that district’s largest school support organization, and an institution that had historically 
trained many of its teachers to create UTR. The NYCDOE, New Visions for Public Schools, and Hunter College took 
a path similar to other residency and alternative certification programs, but their UTR model also diverged in 
important ways: though it was a quicker fix than some traditional preparation programs, UTR’s 14-month span 
stressed the critical importance of extended clinical practice, ongoing training and support, and acclimation to 
urban school environments. The goal was not only to place new, well-trained teachers in high-need schools and 
classrooms, but to keep them there. UTR also linked teacher certification to student achievement by evaluating 
residents often, guiding them through cycles of inquiry to assess student needs, and requiring those who graduate 
from the program to demonstrate that they can move learning forward. Emphasizing collaboration throughout the 
program, UTR also groomed teachers to use data-driven inquiry to lead school improvement. 


Organization of the Chapter 


Special education in many ways tests and amplifies these key aspects of the UTR model. This chapter shares 
findings on the preparation of 65 UTR special educators, and their impact on—and resilience and retention in— 
NYCDOE classrooms. It begins with a summary of findings; each section also includes a set of key points. Section 
1 provides a profile of the UTR special educator and the program, examining what skills and dispositions Hunter 
and New Visions partners looked for or saw in aspiring special educators and fostered in their training. Drawing on 
records from Hunter and New Visions and from survey responses, Section 2 shares findings from residents’ 
program performance and self-assessments of efficacy and practice, where possible noting differences between 
special educators and their peers. Section 3, based on three case studies, examines how UTR special educators’ 
skills and dispositions translate into practice, and whether their practices differ from those of other special 
education teachers. Section 4 reports findings on UTR special educators’ impact on students’ success, based on 
school data and institutional metrics—attendance, credit accumulation, and graduation rates. The final section 
examines retention among UTR-trained special educators, compared to rates among their peers and other NYC 
DOE teachers, including those who entered the profession through different pathways. 


"4 Dan Goldhaber, Lesley Lavery, and Roddy Theobald, June 2015. “Uneven Playing Field? Assessing the Teacher Quality Gap 
between Advantaged and Disadvantaged Students,” Sage Journals 44, no. 5: 293-307. Available at 
http://journals.sagepub.com/doi/full/1 0.3102/0013189X15592622. 
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METHODS 


For this study, we used a mixed methods approach that combined quantitative and qualitative data. Our 


quantitative data ranged from five years of UTR survey data, to data from the city’s Independent Budget Office, to 
data from New Visions and Hunter records. We also cast a wider net than we had during UTR’s five years to 
examine student performance, analyzing credit accumulation figures, Regents passage, and graduation rates 
among special education students. In addition to historical qualitative data collected over UTR’s five special 
education cohorts, and more recent interviews with project partners, we also developed case studies of three 
schools—to explore what skills and practices UTR-trained special education teachers bring to schools and how 
they are received and incorporated into school practice and culture. 


The following questions guided the study: 
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How well prepared are UTR 


or practice emerge in 


special educators? 
P students’ performance? 


other special education 
teachers? 


Partner Reflections 


As background for all our data collection, we interviewed New Visions and Hunter partners and faculty who 
designed the UTR program and the special education coursework and training. During initial interviews, we 
discussed how partners recruited and selected UTR special educators, what skills and dispositions they looked for, 
and what institutional and district policies defined the program. In a second round of interviews, we revisited the 
model’s evolution, discussing specific programmatic changes linked to policy revisions or a growing understanding 
of how best to prepare special educators. 


Resident Data 


Resident Records 


Data on program performance and completion came from Hunter College School of Education’s Offices of 
Residency Programs, Partnership Programs, and Research. Additional data on recruitment and resident 
performance came from New Visions’ Teacher Certification Data Team Warehouse and Process Monitoring 
System. The sample included all residents from five cohorts (those enrolling in 2010-2014) for whom we had data. 


New Visions Assessment Suite 


Scores generated by New Visions’ suite of teacher assessment tools, provided by the New Visions UTR team or 
housed in their Process Monitoring System database, have been the ongoing source of the residents’ performance 
data. These data include scores on the Defense of Learning (DoL), the lesson and unit design and professionalism 
rubrics, and the Danielson rubric (modified for special education). We looked at percentages of residents meeting 
benchmarks for each tool, differences by subject, and performance arcs or successive scores. The numbers of 
residents in the samples varied by tool and recording intervals. 
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Residency Year Surveys, 2011-2015 (Residents and Mentors) 


Rockman surveyed UTR special education residents and mentors, along with their colleagues in other subject 
areas, throughout the UTR project. Although UTR formally ended in 2014, New Visions continued with a sixth 
cohort of residents, who completed surveys in Spring, 2015.'° As part of our annual survey analyses, we created a 
set of scales on efficacy, classroom management and environment, instructional practices, assessment, and school 
culture, along with overall scales on efficacy, best practices, and professional engagement; scales were based on 
factor analyses and reviews of how survey items accounted for variance in survey data. We have used these same 
scales in this chapter to discuss how self-assessments of efficacy, practice, and professional engagement varied 
between special education residents and those in other content areas, and between residents and their mentors. 


The historical survey sample includes 60 special education residents and 58 of their mentors, starting with the first 
special education cohort, in 2010-2011, and continuing through 2014-2015, with the fifth cohort. Table 1 on the 
following page shows responses for both groups, by cohort or year. The average response rate was high—at 89% 
for mentors and 92% for residents. 


Table 1. Historical Survey Data: UTR Special Education Residents and Mentors 


Mentor Resident 
Cohort 2 (2010-2011) 3 12 
Cohort 3 (2011-2012) 14 15 
Cohort 4 (2012-2013) 14 14 
Cohort 5 (2013-2014) 9 8 
Cohort 6 (2014-2015) 18 V4 
*TOTALS/Response Rates | 58 (89%) | 60 (92%) 


Source: REA Annual Surveys 


Although some UTR special education residents taught middle school classes, most—and thus most of the 
historical survey data respondents, around 85% —were high school teachers. Among those, most taught 9th and 
10th graders, in similar percentages of approximately 72% each; smaller percentages taught the upper high school 
grades, with 59% reporting that they taught 11th graders, and 43%, 12th graders. (Totals exceed 100% because 
respondents taught multiple grades.) 


Where appropriate, we compared special education residents’ survey responses to responses from approximately 
70 residents in other subject areas. 


® There were some differences between the UTR model for Cohorts 2-5 and that used for Cohort 6, also Known as UTR 2.0. 
Resident and mentor stipends for Cohorts 1-5 were funded through the UTR grant from the US Department of Education’s Office 
of Innovation and Improvement, while Cohort 6 resident funding came directly from New Visions schools, which helped interview 
and select Cohort 6 residents, who they then hired as part-time teachers of record. These residents were responsible for two 
class periods and required to be at their host schools five days a week, completing their Hunter coursework in evening classes. 
Residents in Cohorts 2-5, by contrast, taught one focus class, four days per week during the school year, and spent one-day 
taking classes at Hunter. These variations in the model may account for some SY 2014-15 differences noted in this report. 
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Comparison Hiring and Retention Data 


Additional data on general and special education hiring and retention came from the Independent Budget Office 
(IBO) of the City of New York, a publicly funded agency that provides nonpartisan information. In addition to issuing 
annual publications on different topics, including those related to public schools, IBO also responds to requests for 
specific data, and, for this study, IBO supplied data that we combined with data from New Visions to explore 
retention among UTR-trained special educators, compared to city-wide rates. 


Student and School Performance Data 


The institutional data used in our analyses of UTR’s impact on special education students, provided by New 
Visions, was drawn from the 70 high schools in the New Visions network. We sorted schools by concentrations of 
UTR-trained special education teachers, and, aggregating results to the school level, used this school-level 
concentration index as a predictor value. Focusing on 11th and 12th grades, as the most reliable points to assess 
student progress, we examined credit accumulation and graduation rates for IEP students. We examined two 
conditions—UTR vs. non-UTR schools, and by the concentration of UTR-trained teachers. We also explored 
publicly available data that, for example, compares graduation rates among special education students to overall 
school rates. 


The sample sizes for our school and student-level analyses varied. Within grades, sample sizes were similar, and 
the non-UTR groups were typically somewhat smaller than the UTR groups. For the UTR vs. Non-UTR 
comparisons, there were, for example 374 11th graders and 346 12th graders in 11 schools in the UTR group, vs. 
223 and 250 respectively in 13 schools in the non-UTR group. Tables 2 and 3 show the numbers of schools and 
students included in each condition for the analyses by concentration of UTR-trained teachers. 


Table 2. UTR vs. Non-UTR 


Condition UTR (11 schools) Non-UTR (13 schools) 
Students 11th grade 12th grade 11th grade 12th grade 
by grade 374 346 223 250 


Table 3. UTR Concentrations 


UTR (13 schools) Non-UTR (11 schools) 
Condition 
Low (4 schools) Medium (3 schools) | High (6 schools) 
Students 11th grade 91 95 188 250 
by grade 12th grade 78 78 190 250 


Case Studies 


School Demographics 


We developed three case studies to portray how UTR-trained special education teachers applied skills and 
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practices, what influence their training in diagnosing and meeting the needs of special education students had on 
ICT team teaching and levels of school collaboration, and how their approach to their work compared to that of 
teachers trained in other traditional or alternative certification programs. 


Criteria for site selections included the configuration of special education services; the concentration of UTR-trained 
special educators and other subject-area teachers; and the school’s demographic and performance profile, 
including the percentage of students with IEPs. All three schools have diverse student populations, though School 
A, the largest school, has a higher population of Asian students (84% vs. 2% at School B and 1% at School C). 
Percentages of Hispanic students at Schools B (59%) and C (72%) exceed that of School A (29%); there is less 
variation in the percentages of Black students, which ranges from 26% to 35%, and White students, at 4%, 2% 
and 0%. School A has around half as many students with disabilities— 10% vs. 22% and 25% for School B and C; 
School A percentage of English Language Learners was similar to School C’s (14% and 19%), both higher than 
School B’s (7%). The economic need index for the three schools was 59% (A), 72% (B), and 86% (C). (See Figure 
1. See also Section 3, starting on p. 37, for additional information on case study schools.) 


Figure 1. Case Study Profiles at a Glance 
Student Ethnicity 
72% ™School A 


59% 
© School B 


34% = School C 
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Students with Disabilities English Language Learners Economic Need Ingex 
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UTR Concentration 


One of the goals of the case studies was to see if UTR special educators brought demonstrably different skills to 
schools. In selecting sites, we looked at the concentration of UTR special educators to see if having more UTR- 
trained special educators made a difference. Because all UTR residents receive similar training, and bring a similar 
set of skills to schools, we also looked at the overall numbers of UTR residents hosted and hired by schools in our 
pool of possible case study candidates, all in the New Visions network. The three schools selected represented a 
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high, medium, and low concentration of UTR-trained special educators, and high, medium, and low concentrations 
of all UTR-trained teachers (see Table 4). 


Table 4. UTR Concentration in Case Study Sites 


School # UTR Sped # UTR non-Sped # UTR Sped # UTR non-Sped UTR 
Residents Hosted Residents Hosted Grads Hired Grads Hired Concentration 
School A 16 16 5 3 High 
School B 1 1 9 1 Peaivin 
School C 0 0 1 0 on 


= School A, the high concentration school, and the largest, had hosted 32 residents over Cohorts 1-5, half 
of whom were special education residents; they had hired five of those residents (Cohorts 8, 4, and 5). 

= Over the same period, School B, the medium concentration school, had hosted 2 residents, 1 of whom 
was a special education resident, 1 of 2 special education residents hired by the school (Cohorts 3 and 5). 


= School C, our low concentration school, had not hosted residents but had hired 1 special education 
resident (Cohort 2). 


Interviews and Focus Groups 


We visited each of the case study sites, interviewing administrators, special education coordinators, and pairs or 
groups of special education and content area teachers. Interview questions varied to some degree from site to site, 
but we used structured protocols for each site. We also observed classes at schools A and B, talking informally 
with teachers about their plans and collaborative activities. Some teachers also shared lesson plans and handouts. 


Case Study Teacher Surveys, 2015-2016 


We invited special education and content area teachers in the three case study sites to complete brief surveys. The 
surveys focused on areas we discussed in more depth with administrators, special education coordinators, and 
teachers: teachers’ preparation for meeting the needs of special education students, schools’ programs for 
students, and collaboration among teachers. The goal was to gather uniform data across sites, and invite feedback 
from a wider range of teachers, especially on team teaching, at each school. A total of 30 teachers from the three 
case study sites completed surveys; respondents included both UTR- and non-UTR-trained teachers. The majority 
of respondents, overall, taught 9th and 10th grade students (80% and 67%, respectively); 40% taught 11th and 
12th graders (24 of the 30 respondents taught more than one grade level). Half of the respondents had been 
teaching at their current schools for three years or less—a fifth of those were new to their schools. Forty percent 
had four to 10 years’ experience at the case study schools, and another third, 11-15 years (See Figure 2). 
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Figure 2. Case Study Survey Respondents’ Years Teaching at Current School 
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Although we invited both special education and content area teachers to take the survey, the respondent group 
was not always balanced and groups varied by school. The overall respondent group included more special 
educators (73%, N=22) than content area teachers (27%, N=8), although the subjects taught by the latter group 
spanned the core high school subjects (1 math, 2 science, 3 English, and 2 social studies). At School A, no content 


area teachers completed the survey. (See Table 5.)'° 


Table 5. Case Study Survey Respondents by School 


Special Education Teachers : Content Area Teachers 


- ‘School A (N=17) 17 (7% =| 0 . 
School B (N=3) 1 (6%) | 2 (25%) 
School C (N=10) | 4 (18%) 6 (75%) 
TOTALS 22 (73%) | 8 (27%) 


Percentages of traditionally trained teachers vs. those who entered the profession through alternative certification 
were similar, overall and by subject: of the 30 survey respondents, 16 entered the profession—nine, through the 
New York City Teaching Fellows program; six, through UTR; and one through another program. The other 14 
teachers entered teaching through traditional preparation programs. (See Table 6.) 


Table 6. Case Study Survey Respondents by Preparation 


| Traditional | Alternative Certification 
Special Education (N=22) 10 (46%) 12 (54%) 
Content Area (N=8) a 4 (50%) | 4 (50%) 
TOTAL 14 (47%) 16 (63%) 


ie The percentages in Tables 5 & 6 indicate the proportion of the survey sample, not the proportion of special education or 
content area teachers in the schools. Although a few resource room teachers did not complete surveys, we had a high response 
rate for those who teach in ICT classrooms. Those potential numbers can still vary some due to classroom needs, ICT 
assignments, and team pairings. The possible number of content area respondents can vary or shift even more, again depending 
on pairings and the fact that special education teachers may rotate depending on content-area needs or schedules. 
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Data Analysis 


Survey Data 

We used basic descriptives and frequencies to analyze teacher survey data, disaggregating data by cohort, subject 
area, or respondent group to examine differences between, for example, residents and mentors, special educators 
and content area teachers, or UTR-trained teachers and those trained in other programs. In most cases, the 
sample numbers were too small to make meaningful comparisons based on statistical significance. 


Open-Ended Survey Items and Focus Group and Interview Data 


For open-ended survey questions, we conducted content analyses, first reviewing responses for recurrent themes, 
then coding responses. We focused on categories or constructs in the survey items themselves, looking at 
confidence in preparation, confidence in and use of assessment strategies, collaborative activities, and school 
culture. To check for inter-rater reliability, we discussed coding strategies and categories and normed as needed. 
We also analyzed open-ended survey responses and focus group and interview data—for the case study sites and 
the interviews with project partners — by looking for recurrent themes, within and across groups. 


Student Performance Data 


As noted above, we drew our student and school samples from over 70 NYC DOE schools in the New Visions 
network and focused on grades 11 and 12 as the best gauge of students’ high school progress. The numbers of 
students in each analysis varied. (See individual tables in Section 4, starting on p. 51.) Under two sets of conditions, 
we analyzed student- and school-level outcomes—attendance, credit accumulation rates, four- and five-year 


graduation rates, and the numbers of Regents, out of five, passed at 55 and 65. 
1. The first condition, with only special education students, included two categories: UTR and Non-UTR. 


2. The second condition, with special and general education students, designated schools as having low, 
medium, and high concentrations of UTR special educators, or no UTR-trained teachers. Our concentration 
index used the number and percentage of UTR-trained teachers at schools as well as length of involvement. 


For student-level analyses, students in the UTR group had to be attending a school with at least one UTR-trained 
special education teacher during SY 2014-2015. To be included in the non-UTR group, students had to be 
attending a school where there were no UTR-trained teachers—special or general education—at any point over the 
past four years. (The rationale was that UTR-trained teachers use similar strategies to diagnose and meet students’ 
needs, so a UTR-trained content area teacher could have an effect on special education students.) 


For the school-level analyses, schools in the UTR group had to have at least one UTR-trained special education 
teacher in 2014-2015 and at least 10 special education students in the specified grade level. For the non-UTR 
designation, the school could have no UTR-trained teachers of any kind (special ed or general ed) at any point over 
the past four years, and had to have at least 10 special education students in the specified grade level. 


For both sets of conditions, we conducted school-level analyses to explore whether having UTR teachers on staff 
narrows the gap in institutional outcomes between special education students and their general education peers. 
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SUMMARY OF FINDINGS 


SIDENT AND PROGRAM PROFILES 


Crafting a special education model required dialogue, balance, and continuous improvement, all of 
which ultimately strengthened both the model and the Hunter-New Visions partnership. As UTR partners 
created their special education model, schools were adjusting to the NYC DOE’s new special education policies 
and ICT configurations. This required ongoing conversations to ensure that Hunter coursework and New Visions 
support focused on the skills and knowledge residents needed. Clinical experience gave residents a context for 
coursework, and the experience they brought to Hunter classrooms helped build a bridge between theory and 
practice. In designing preparation and defining best practices, it was also important that partners keep the pace of 
change in schools in mind and not get ahead of where schools were. 


The mindsets residents brought to UTR—and further developed during their clinical year—helped them 
acclimate to urban schools. UTR special education residents have been described having an idealistic outlook 
and social justice slant, tempered by an understanding of what they can and cannot change—all invaluable skills 
for urban school educators. Though UTR special educators share an idealism and humanitarian outlook with many 
who go into the field, their extended clinical experience grounded that idealism in real classroom experience. 
Compared to other special educators and residents in other subjects, they see more quickly “that rigid doesn’t 
work,” which can make acclimation to urban classrooms easier. 


UTR training leveraged this idealism coupled with flexibility, equipping residents not just to acclimate to 
urban schools but also contribute to them. UTR training built on the flexibility often spotted during candidate 
screening, helping residents learn how to assemble data and what to do with it as they operationalized instructional 
strategies. Teachers and administrators in placement schools praised this skillset around using data for 
interventions, and credited residents with helping them meet the needs of special education students. 


Other demographic factors also made UTR special educators a good fit for urban schools. Research 
suggests that backgrounds or ethnicities similar to students’ and a familiarity with their urban environment can 
strengthen teacher-students bonds. Twenty-nine percent of the 65 residents trained in Cohorts 2-6 are African- 
American or Hispanic, groups often underrepresented in the teacher corps but the predominant populations in 
schools. Close to half of the UTR special educators also attended universities in New York City. 


Continuing to emphasize collaboration helped mitigate the challenges of content-area pairings and 
gave residents more parity. There are school factors that can make UTR special educators feel less effective. An 
ongoing challenge is that they may co-teach in content areas that do not align with their strengths. Schools 
typically need math, perhaps science teachers, but special educators are more likely to have humanities 
backgrounds. As the program matured, an increased focus on collaboration helped address the challenge. 


U 


EPARATION & PERFORMANCE 


Attrition rates that are higher during the residency but lower in early career years may suggest that 


special educators, or program staff, realize earlier that the profession is not the right fit. Special 
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education is a challenging path for any novice teacher, and the requirements for special educators, including 
certification requirements, can present additional challenges. The demands may be reflected in the fact that, over 
five cohorts, five special education residents withdrew from the program and nine were counseled out, for an 
attrition rate of 17 percent, compared to 10 percent among other subject-area residents. Retention data, 
however, trends in the opposite direction, suggesting that it may be obvious sooner rather than later that special 
educators have made the right career decision. 


On other measures, special education residents’ performance was on par their UTR and Hunter peers. 
Degree completion rates among special education residents were high, and on par with those of residents in other 
subjects. UTR special education residents’ Hunter GPAs were also high, and within a few tenths of a percentage 
point of other residents and Hunter graduate students. (This was true of special education residents who chose to 
leave the program and those who were counseled out.) Placement rates were at 100%, similar to rates for math 
residents, also a high-need subject. 


UTR special education residents’ performance on the New Visions assessment suite, and self- 
assessments for related skills, followed trends, with a few notable exceptions. Special education 
residents’ Danielson ratings were similar to subject-area peers’ —slightly higher for domains related to classroom 
environment, slightly lower for assessment and questioning domains. Survey responses showed a similar inverse 
relationship in self-assessments and Danielson scores: like subject-area peers, special educators felt more 
confident in the skills for which Danielson ratings edged down. Variations in both trends included higher Danielson 
scores overall for Cohort 6 (whose program model, as described earlier, differed somewhat) but lower self- 
assessments for certain skills. 


Rising Danielson ratings and residents’ and mentors’ self-reports of professional engagement 
suggest a maturing program. In addition to the higher Danielson scores for Cohort 6, disaggregating data 
by special education cohort showed an increase in levels of school collaboration across years, along with 
higher means on a professional engagement scale that clustered activities related to work with colleagues and 
shared decision-making. The trends may reflect an increased emphasis on collaboration in special educators’ 
training and the evolution of the UTR special education model. 


Special education graduates and school coordinators agree that more training in writing and 
monitoring IEPs could help UTR special educators during their residency and induction. Case study 
survey responses and feedback from interviewees indicated that UTR residents, like most new teachers, find it 
challenging to tend to the IEP work increasingly required of special educators and require specific IEP training 
during their residency year. 


RACTICES AND SCHOOL IMPACT 


We found no differences between UTR-trained special educators and teachers prepared in other 
programs in their commitment to serving students and making ICT classrooms work. In the three 
case study sites, where numbers of UTR-trained teachers and special education programs varied, the devotion 
to serving students—and reports of the challenges involved in doing so—varied very little. As a whole, special 
education teachers (both UTR trained and others) did not feel prepared to teach in resource setting and to 
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develop IEPs or 504 plans. UTR special educators felt more prepared than peers to co-teach in ICT classes, 
but like their peers indicated some reservations about preparation. 


Where we did find differences were in how prepared UTR special educators feel to meet the needs 
of students and use formative assessment strategies to do so. Compared to special educators 
prepared in other programs, UTR special educators felt better equipped to: create formative assessments, 
plan instruction around them, and tailor instruction to students’ needs. This confidence combined with a toolkit 
of formative skills and strategies seemed to set UTR-trained group apart. 


Specific skills emphasized in preparation show up clearly in practice. Case studies A and B provide 
evidence that the inquiry, formative assessment, leadership, and collaboration skills that UTR special educators 
gain in their training are visible in their practice as in-service teachers. Content area teachers, for example, 
allude to UTR special educators’ “inquiry mindset,” and say that, compared to differently trained teachers, UTR 
special educators are “better at driving instruction,” and “not timid about making decisions.” Administrators 
observe that they are “willing not just to collaborate, but to try things, back it up.” Interviewees’ comments and 


the case study observations indicate that UTR special educators are able to operationalize pedagogy. 


UTR special educators not only contribute to shared decision-making but also proactively foster it. 
The case studies suggest that the influence that UTR special educators can have is not limited to special 
education classrooms. In part because some assume leadership roles, or, again, are not timid about being 
proactive, UTR special educators can encourage school-wide changes in how teachers view students with 
special needs and how teachers think about assessment. Although our evidence is limited to a handful of 
schools, it suggests that a concentration of UTR special educators or a few UTR trained teachers can have an 
impact. 


PACT ON STUDENTS 


UTR special educators may positively affect special education students’ success and progress in 
school. Although we cannot show clear correlations, data suggested that the instruction and interventions 
UTR special educators bring to schools, and the relationships they forge, may affect student behaviors and 
school performance and progress. Comparisons showed that 11th and 12th grade special education students 
in schools with UTR-trained special educators had higher attendance rates and earned more credits than 
special education students in schools with no UTR-trained teachers. The differences were statistically 
significant. 


There were also indications of an effect for students of color. Findings from similar comparative 
analyses with attendance, credit accumulation, and graduation data suggested a similar predictive value or 
positive impact of UTR special educators on Hispanic and African-American students, though the differences 
were not statistically significant. 


Comparisons of institutional data in schools with varying concentrations of UTR-trained special 
educators also suggested a positive effect, but no clear trends emerged. Having UTR-trained teachers 
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may lead to higher attendance, credit accumulation, and 5-year graduation rates. What the data did not 
confirm was that the higher the concentration the greater the effect. 


= Results from analyses designed to explore whether UTR-trained special educators closed the gaps 
between special and general education students were mixed. Using the same gauges of school 
progress, our final analyses looked at differences between the two groups of students. Although some gaps 
narrowed, indicating that special education students were catching up to their peers in credit accumulation 
and graduation rates, there were instances in the schools without UTR-trained special educators as well as in 
those with UTR teachers, in different concentrations, where the gaps narrowed. 


ay 


NTION 


=» UTR-trained special education teachers have a 93 percent retention rate, just above the overall UTR 
rate of 91 percent. In line with our previous examinations of retention data, the rates remain high. Sixty of the 
64 trained teachers are still teaching. (One of the original 65 is deceased.) Two have left the classroom, and the 
status of two others is classified as “unknown.” Of the 60 still teaching, 58 or 97 percent are teaching ina NYC 
DOE public or New Visions Charter school. 


= Retention for UTR special educators after one year is higher than rates for special educators 
trained in traditional and other alternative certification programs. Data from the NYC IBO indicate that, 
after one year, UTR had higher retention and no declines. The UTR rate for special educators in their second 
year was 93% compared to 49% for Teach for America, 72% for Teaching Fellows, and 77% for traditional 
pathways. 


= Retention rates among UTR special educators exceed rates among special educators city-wide. 
Attrition rates for special educators hired in New York City’s public schools are actually lower than rates among 
high school teachers overall, but attrition for both groups increase year by year in teachers’ first years in the 
classroom. Rates for UTR special educators increase slightly after two and three years, but by smaller margins. 


= UTR special educators, compared to their UTR peers, tend not just to stay but to stay put. Although 
all retention figures are high, mobility data indicate that special educators are less likely to change schools. 
Thirty-two of the 44 special educators for whom we have data—73%—are still teaching where they started. Of 
the 12 teachers (27%) who changed schools, all did so only once. We do not have sufficient data for 
attribution, but stability among special educators may be due to skills and characteristics that are part and 
parcel of their training and chosen field: more experience working—and succeeding—with a wide range of 
students; adaptability; and a better understanding of how novice teachers might advocate for certain practices. 


= UTR special educators engage in and foster activities linked to higher teacher retention. Feedback 
from the case studies indicate that when UTR special educators apply their training in inquiry methods and 
assessment strategies the benefits accrue not just to their special education students but also to other special 
educators, the content teachers they team with, and the school as a whole. They can foster collaboration, a 
positive school climate, leadership among teachers, and accountability for students’ success—all of which 
contribute to retention. 
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On a broader level, UTR special educators can have a positive effect on how schools manage 
change. Although there were challenges, the timing of the launch of the UTR special education program was 
fortuitous: Because UTR special educators began teaching in NYC schools as those schools were adjusting to 
new policies, partners could target training, and newly trained special educators could employ skills that helped 
schools adapt and carry out changes in ways that supported both students and teachers. 
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Section 


Profiles & Preparation 


By training 65 new special educators, UTR helped fill critical needs in 
New York City’s public high schools. Teacher shortages were due in 
part to attrition—special education is a demanding subject, perhaps 
more so in urban school environments—but attrition wasn’t the usual 
suspect or sole factor. The overall numbers of special education 
teachers in the city’s schools were actually increasing, even as the 
overall numbers of teachers were still on the decline.” What had also 
increased were the numbers of students requiring special education 
services. There wasn’t a sudden, inexplicable uptick in the numbers of 
children with disabilities, but new findings and heightened awareness 
about attention deficit and autism spectrum disorders, and socio- 
emotional problems that can affect learning meant that more children 
were being identified as having special needs. The rolls also grew 
because of demographic changes, which increased the numbers of 
students with language and literacy needs, and economic downturns, 
which put already vulnerable students at greater risk of failing in school. 


The NYC DOE had responded to growing needs by revising policies 
and hiring more special educators, in some cases issuing temporary or 
provisional licenses. The district had also extended outreach to 
parents, with information and invitations to work with school teams to 
develop Individualized Education Programs (IEPs), the legal document 
that spells out a child’s educational goals and services. Still, according 
to a 2005 NYC DOE report, schools were not meeting the needs of 


At 47%, special education residents 
were UTR’s largest single contingent 
among the 138 residents completing 
UTR training in Cohorts 2 through 6. 


61% of the UTR special education 
residents in Cohorts 2-6 were career 
changers; 45% of those brought 1-5 
years of experience to the program. 


30% were from underrepresented 
groups; their undergrad institutions 
were more often private, but more likely 
to have been in NY, with 42% in NYC. 


Hunter faculty members say UTR 
special educators are idealistic but 
grounded, & learn sooner than their 
peers that “rigid doesn’t work.” 


Adapting the special education model to 
changing NYC DOE policies, student 
populations, & schools adjusting to both 
required & strengthened dialogue 
between partners. 


A focus on collaboration can help 
prepare special educators for the 
challenges of ICT classrooms. 


students with disabilities.'° As the district continued to revamp programs, federal policies also changed, as the 


2006 IDEA reauthorization required all public elementary and secondary special education teachers be “highly 


qualified”; in addition, the legislation described alternative routes to certification. '? 


UTR partners added special education to its list of subjects in 2010-2011, with its second cohort of residents. They 


entered a field that presented opportunities— increases in funding,?° and the newly launched ICT model, which 


17 Over the 12-year period studied, as the number of general education teachers fell by more than 9,100 to 54,778, the number 
of special education teachers grew by more than 5,400 to 18,595. The NYC IBO figures and review available at: 


http://www. ibo.nyc.ny.us/iboreports/201 4teacherdemographics.pdf. 


q Thomas Hehir, et al. “Comprehensive Management Review and Evaluation of Special Education” (Sept. 2005). Available at 


https://www.uft.org/files/attachments/hehir-report. pdf. 


19 See http://idea.ed.gov/explore/view/p/%2Croot%2Cdynamic%2C QaCormer%2C2%2C. 
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moved many students with disabilities out of restrictive, pullout classrooms into general education ones—a change 
praised by a second 2012 study of the new city-wide policies.2' There were also challenges, as schools adjusted to 
new policies, procedures, and faculty configurations. This section looks at the 65 special educators trained, 
credentialed, and placed through UTR, and program’s adaptation to a changing and challenging environment. 


RESIDENT PROFILES 
At 47%, special education residents were UTR’s largest single contingent among the 138 residents who completed 


UTR training in Cohorts 2 through 6. Percentages ranged from 37%, or 13 out of 35 residents in Cohort 2, to 74%, 
or 14 out of 19 residents in Cohort 6.2" (See Figure 3.) 


Figure 3. Special Educators and Other Residents, 
Cohorts 2-6 


22 
15 ud 15 
13 14 14 14 
9 
Lal . 


UTR Cohort 2 UTR Cohort 3 UTR Cohort 4 UTR Cohort 5 UTR Cohort 6 


™Special Education Residents All Other Residents 


Who Are UTR Special Educators? 


The selective UTR screening and admissions process was designed to identify the most promising candidates. Like 
all those aspiring to become UTR residents, special education applicants had to clear several hurdles, including 
meeting the requirements for New York City Teaching Fellows and Hunter’s graduate school entrance 
requirements, which stipulated that applicants have a bachelor’s degree from an accredited institution with a GPA 
of at least 3.0, or a master’s degree with a GPA of at least 3.5. In additional rounds of screening, New Visions 
looked for talent, diversity, and a combination of the experience, idealism, and grit that would equip candidates to 
work with teachers and students in high-need urban schools and ICT classrooms. There were also efforts, based in 
part on research suggesting that teachers form stronger bonds with students when their backgrounds and 


ethnicities are similar, to attract residents already familiar with urban environments. 


ed NYCDOE spent $1.26 billion on special education instruction in 2009-10; $1.3 billion in 2010-11; $1.5 billion in 2011-12; and 
$1.6 billion in 2012-13. See IBO Report, 2014. 

ml “Educating All Students Well: Special Education Reform in New York City Public Schools,” Fund for Public Advocacy (August 
31, 2012). 

22 In 2012, a Cohort 3 resident was tragically killed during Hurricane Sandy. In this report, when we refer to the numbers of 
special educators trained, we use the total figure of 65. In the discussions of attrition and retention, we use a total of 64. 
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Background and Demographics 
Across the five cohorts (2-6), the professional experience that UTR special educators brought to classroom differed 
only in small ways from that of their cohort peers. Compared to residents in other subject areas, a smaller 
percentage of special educators—31% vs. 40% —came to the program straight out of college. Similar percentages, 
with proportions reversed, had five years’ experience or less. In addition to career changers with limited time in 
other pursuits, there were those in both groups 


Figure 4.Years of Professional Experience, 
with more experience under their belts, including UTR Cohorts 2-6 
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Figure 4.) Prior experience figures deviated more ‘severe se 
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professional experience. 


The ethnicity of special education residents also largely mirrored that of UTR cohorts as a whole, with over half of all 
residents—57% of special education residents and 52% of all other residents—identifying as White/Caucasian. 
Similar percentages of special educators and all other residents identified as Hispanic/Latino and Black/African 
American, for totals of 28% and 29% respectively from underrepresented groups. The only ethnicity category that 
had notably lower representation among special education residents compared to all others was Asian. The 
percentages for African- American and Hispanic teachers in the city’s high-poverty secondary schools exceeded 
UTR figures by four to six percentage points; figures for Asian teachers were higher in the UTR group than in the 


broader city population. (See Figure 5.)*° 


23 1BO Schools Brief: A Statistical Portrait of New York City Public School Teachers, p. 5. Available at: 
http://www. ibo.nyc.ny.us/iboreports/201 4teacherdemographics.pdf, NYC 
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Figure 5. Ethnicity: UTR Special Educators vs. Other Residents & NYC DOE 
High-Poverty Secondary Teachers 
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Other demographic data pointed to some differences between UTR special education residents and those who 
chose other subject areas. Although there were, for example, more female residents than males in other subjects, 
as there are in the profession as a whole, the gender division was more striking for special educators, with over a 
20 percentage-point difference. Across the five cohorts, 82% of special education residents were female, 
compared to 60% of residents in all other subject areas. The percentages for residents in other subject areas were 


more similar to citywide figures for teachers in high-poverty secondary schools. (See Figure 6.)** 


Figure 6. Gender Breakdowns: UTR Special Educators vs. Other 
Residents & NYC DOE High-Poverty Secondary Teachers 
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UTR special educators were also more likely than other residents to have obtained their undergraduate degrees 


from a private institution—in nearly inverse proportion. The margins were smaller, but special education residents 


ae IBO Schools Brief: A Statistical Portrait, p. 5. 
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were also more likely to be educated locally: 51% of special education residents, vs. 45% of all other residents, 
earned their undergraduate degrees in New York. Of these, 42% of the special education residents and 27% of alll 
other residents obtained their undergraduate degree from an institution in New York City. (See Figure 7.) 


Figure 7. Educational Backgrounds of Residents 
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Skills and Dispositions 


Hunter faculty members observe certain characteristics they say UTR special education residents share. In addition 
to being “high-caliber students,” they are, by and large, “very committed, interested, dedicated.” They “know 
themselves socially and emotionally,” and possess an “ability to empathize far above others,” a “heightened 
sensitivity to meeting students’ needs,” and a “social justice slant.” 


These traits may point to some finer distinctions between UTR special educators and other aspiring teachers, but 
they don’t necessarily set UTR special educators apart from other special educators—those who choose the field 
are typically a dedicated, empathetic group. What does seem to set UTR special education residents apart, 
according to one faculty member, is that they seem “more grounded,” more “clear-eyed” about “what they can and 
can’t fix.” It may be that they gain these skills or dispositions from their extended clinical experience—and acquire 
them more quickly, out of necessity. Residents have often been described as being high achievers, as one Hunter 
faculty member observed, “grieving about B’s.” Even compared to UTR peers, special educators, he also 
observed, tend to learn sooner that “rigid doesn’t work.” 


PROGRAM PROFILE 
How Are UTR Special Educators Prepared? 


Initial Screening and Partnership Efforts 


Residents’ year-long clinical experience in real schools and ICT classrooms gives them a head start on skills more 
typically acquired during novice teachers’ induction years, but other parts of their training are specifically designed 
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to give them both skills and flexibility. Through the collaborative inquiry work that is paramount in New Visions 
training, all UTR residents learn to cycle through a process whereby they assess students’ needs, design or adapt 
instruction based on feedback, retry, and reassess. In an ICT setting where special educators collaborate with 
content teachers to adapt lessons to a range of students’ needs, they may learn to pivot easily, quickly, and often. 


According to New Visions current Director of Teacher Residencies, Marisa Harford, as the project team refined a 
screening process, they focused on these skills earlier, requiring aspiring special educators reflecting on demo 
lessons what they would do with feedback to “reach students” and “make content accessible.” An additional skill 
that became part of their screening criteria involved not just the skills needed to adjust lessons, but also “initiative 
taking”: screeners looked for candidates who would also be “pro-active,” likely to “advocate for themselves.” 


“Quality” was one operative word in the TQP program; “partnership” was the other. Getting more teachers in urban 
classrooms, keeping them there, ensuring they could be highly effective—this required a collective effort, which was 
not without its own challenges as partners responded to research on best practices, new city policies, and 
feedback from residents and mentors. 


Incorporating the “needs and obligations” of all partners was, according to David Connor, Director of Hunter’s 
Special Education program, a “dialogic” process. Needs changed as the city’s policies took shape and special 
educators assumed residencies in schools that were adjusting to new policies. As a group, partners engaged in an 
ongoing conversation about what special educators needed to know, and when, and whether they needed to 
experience all facets of special education to be effective. Sometimes stakeholders varied and agendas differed: 
there could, for example, be a conversation between partners and the NYC DOE about ICT requirements, between 
faculty and New Visions about shared and proprietary responsibilities, or between Hunter faculty devising a new 
course and state- level approval boards. As the model evolved, the goals remained lofty and practical: for Hunter, 
to “arrange coursework and training in a matrix that makes sense,” and, for New Visions, to figure out “what’s 


working” and “what needs to be worked on more.” 


Hunter Coursework 


The course map for Hunter’s graduate Adolescent Special Education Program is similar to those for other aspiring 
teachers. The program, through which residents earn a generalist degree for grades 7-12, and, upon graduating, 
are eligible for New York state certification in Students with Disabilities, Grades 7-12, is built around methods, 
adolescent development, assessment, classroom management, and practicum experience. Coursework includes: 


Issues and Practices in Educating Students with Disabilities 

Assessment of Students with Disabilities 

Classroom Management in Special Education and Inclusive Settings 

Inclusive Instruction in General Education Classrooms for Students with Learning and Behavior Disorders 
Issues in Teaching English Second Language Learners with Special Education Needs 

Methods of Teaching Reading for Adolescents with Learning Disabilities 

Supervised Clinical Teaching of Adolescents with Disabilities: Part | 

Adolescent Development and Learning: Grades 7-12 


According to Department Chair Connor, the special education program is in some ways an “artifact of the 
shortages” and the need to train teachers to work with an increasingly diverse student population. No one 
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questioned that residents’ preparation should reflect the special needs of the students they would eventually serve. 
But covering the possible learning disabilities and disorders called for a “large umbrella,” and determining, for 
example, what should—or could—be included in a course entitled “Diverse Learners” required negotiation. 


One aspect of what Connor called a “dialogic” process was the proverbial division between theory and practice. 
Hunter’s program, he explained, has always been “predominately practitioner focused,” with emphasis on “how to 
plan the lesson, how to base lessons on assessment, how to conduct formative and summative evaluations.” In 
some ways a residency program fit well with Hunter’s approach. Unlike other graduate students, residents could 
share actual school experiences with practicum faculty. There was also some natural overlap, as that between 
reading and literacy courses. Professors had traditionally worked together, and when classes were largely 
composed of UTR ELA teachers or special educators who shared a common need—for example, to learn about 
vocabulary practices—it was possible to change a syllabus accordingly. Hunter’s Learning Lab, in which novice 
special educators worked one-on-one with students, provided another opportunity to blend theory and practice. 


Defining best practices required more dialogue. Not only were schools adapting to new policies, but the field of 
special education itself was in flux, each area, according to Connor, had its own “little pots of thinking.” Hunter 
Special Education Professor Kate Garnett worked with then Director of Teacher Residencies, Roberta Trachtman, 
Harford, and the New Visions team to design a framework for the UTR special education program around ideas of 
best practices in special education. The challenges didn’t end with defining best practices, but continued with 
enacting them and setting realistic timeframes and expectations for residents. 


New Visions Training and Support 


The first part of readying residents was deciding how to start and sequence residents’ clinical year training. As the 
UTR special education model evolved, New Visions’ asked the same set of questions: What should be prioritized in 
summer training? What should be included in the residency essentials? What kinds of “turnkey” information do 
residents need on that first day? Questions also arose about whether to differentiate training for special education. 
All novice teachers need certain essentials and instructional strategies, but those, says New Visions’ current Deputy 
Director of Residencies, Rachelle Verdier, may look different when viewed through a special education lens. In 
learning about small group work, for example, special educators may take different needs into account in grouping 
or proctoring students. All residents need guidance in how to liaise with parents, but for special educators 
interactions may involve creating IEPs, demanding for even experienced teachers. To address these needs, New 
Visions scheduled breakout sessions during the summer training that focused on IEP writing, the city’s web-based 
Special Education Student Information System (SESIS), and ICT co-planning and co-teaching. There were also 
considerations related to synching New Visions support with Hunter coursework and finding natural synergies. 


Other responsibilities that fell to New Visions involved determining how to support residents in schools where 
special education configurations varied. Even though partners aligned preparation to new policies, schools needed 
time to adjust and comply. According to Harford, at the start, most schools were “on the cusp,” just beginning to 
shift to ICT classrooms and team-teaching. Over UTR’s five years, residents experienced challenges right along 
with schools. Some were a matter of school structures: teachers who needed to plan together lacked common 
planning time. Serving more students strained schools already under-resourced, under-staffed. Even as the city 
increased resources with the launch of SESIS, teachers worried that bookkeeping took time away from students. 
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Other challenges came from defining roles and responsibilities. Both mentors and content area teachers were trying 
to learn new roles and, according to Verdier, get a sense of “what collaboration looks like.” While the UTR program 
was advocating for special education residents to take on a co-teaching role, some schools were still in a “one 
teach, one assist” mode, expecting special education residents to be that extra set of hands. 


Harford and Verdier say that team teaching has become more routine. What still complicates resident/mentor 
pairing is that mentors are special educators but residents team-teach with content teachers—and not always in 
classes where their mentors team-teach. Although, to be admitted to Hunter, residents have to have an 
undergraduate liberal arts or sciences degree and an appropriate distribution of content courses, they may be co- 
teaching in an area they don’t feel strong in, or that doesn’t “play to their strengths.” Special education residents 
tend to have backgrounds in the humanities, but the need is in science and math. In an ideal situation, residents 
have the support of two teachers during their apprenticeship. If that is not the case, or if the content area teacher is 
unaccustomed or even averse to planning ahead, co-teaching can be challenging; more important, adds Verdier, it 
is “hard to feel effective.” 


Ongoing Partnership Challenges and Opportunities 


Case study findings and retention data suggest that special educators’ training helps them succeed—and stay—in 
the classroom. Partners continue to explore ways to ensure both. According to Trachtman, the focus is on 
collaboration and other ways to embed ICT practices in residency essentials, or adding a course on team-teaching 
to Hunter coursework. Connor also alluded to more “dialogic negotiations” about ways to give special educators 
more “standing and parity” in content collaborations. Partners continue to blend Learning Lab work, school 
practice, and the Defense of Learning, part of the New Visions assessment suite that requires residents to 
demonstrate that they can successfully address students’ learning needs, mid-year, with a single or small group of 
students, and again at the end of the year with larger groups. At the writing of this report, dialogic negotiations have 
achieved the assessment trifecta of the DoL. Learning Lab, and EdTPA, the new state-mandated certification exam. 
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Section 2 


Performance « Readiness 


This section shifts to how well UTR’s_ special 
educators measure up as “high-quality” instructors. 
Findings are based largely on program measures, 
including residents’ completion and placement 
rates, Hunter College GPAs, and performance on 
the assessment suite developed by New Visions to 
gauge residents’ growth over the course of their 
clinical year. The section also shares residents’ self- 
reports of readiness. 


Here, as elsewhere, we have looked for points of 
comparison: how special educators’ completion and 
placement rates, across five cohorts (2011-2015), 
compare to those of UTR content-area peers; how 
their graduate school performance compares to 
other Hunter students’; and how their  self- 
assessments compare to self-reports from their 


mentors and peers. 


The discussion of the New Visions assessment suite 
data is based on benchmarks and common metrics, 
including the Defense of Learning, the 
professionalism and unit and lesson design rubrics, 
and the Danielson Framework. We looked for ways 
to triangulate these findings, through self-reports of 
confidence in practices assessed by the suite of 
tools, and through comparisons to levels of 
confidence reported by peers and mentors. We also 
referenced trends we had noted in past analyses, 
and ways self-reports from special educators 
mirrored or departed. 


Over 5 cohorts, 82%, or 65 of the 79 UTR special 
educators who enrolled completed their 
residencies. 


Attrition rates among special educators were a 
little higher than peers’, or 17% vs. 10%: 9 
special education residents were counseled out; 
5 withdrew. 


UTR special educators’ on time degree 
completion rates & GPAs were on par with their 
UTR peers & other Hunter special education 
grads. 


Special education, along with math & science, the 
3 areas where demand was highest, had the 
highest placement rates—all at 100%; English 
rates were still high, but 13 percentage points 
below other subjects. 


Special educators performed on par with peers 
on the New Visions assessment tools. Most met 
benchmarks, with percentages increasing from 
Cohort 4 to Cohort 6. Special education residents 
across cohorts had high rates of professionalism 
throughout the school year. 


Like other residents, special educators tended to 
score highest on Danielson domains related to 
classroom environment— engaging students, 
creating a culture of learning, managing 
behavior—& lowest for instructional domains— 
using assessments & questioning. 


Cohort 6’s Danielson scores were higher than 
previous cohorts, though self-reports of 
confidence were lower. Their self-reports of 
collaboration showed an increase over previous 
cohorts’, with levels of professional engagement 
that matched Cohort 6 mentors’. 


Our final set of comparisons included a look at how UTR special educators’ self-reports of professional 


engagement and confidence in collaborative practices, which provided an additional lens through which to view 


their readiness for ICT classrooms. 
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PROGRAM PERFORMANCE 
How Do UTR Special Educators Compare to Peers? 

UTR partners agreed that special education attracts a certain kind of aspiring teacher—and that special educators 
need certain kinds of skills and resilience to succeed. Average residency and degree completion rates, on par with 


peers’ rates, suggest that the program’s screening, training, and support mechanisms worked well. Data may also 
suggest that, with special educators, it is easier to see, sooner rather than later, if the fit is right. 


Residency Completion and Attrition, by Cohort 


From Cohort 2 (2010-2011), with the first special education cohort, through Cohort 6 (2014-2015), 65 or 82% of 
the 79 special education residents who enrolled in the residency program completed it. Completion rates ranged 
from 69% in Cohort 5, to 100% in Cohort 6. (See Figure 8.)° 


Figure 8. Special Education Residency Completion By Cohort 
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Source: New Visions Data Warehouse 


Residents sometimes choose to withdraw from the program when the fit is not right, but, as UTR evolved, project 
partners also developed intervention strategies through which struggling residents received extra support, 
coaching, and plans to address specific problems. In some cases, problems are remedied; in others, residents are 
counseled out. Over five cohorts, five of the 14 special education residents who did not complete the program 
withdrew, and nine were out-counseled. Cohort 4 had four residents out-counseled, the largest number of 
residents in any Cohort; Cohort 5, with 3 of the 13 residents being counseled out, had the highest rate, at 23%. 
Residents withdrew at similar rates across Cohorts 2 through 5, with one or two residents withdrawing from each 
cohort. (See Table 7.) 


Table 7. Special Education Residency Completion by Cohort, 2-6 


Completed Residency Out-counseled Withdrew Total 
UTR Cohort 2 13 87% 1 7% 1 7% 15 
UTR Cohort 3 15 83% 1 6% 2 11% 18 
UTR Cohort 4 14 74% 4 21% 1 5% 19 
UTR Cohort 5 9 69% 3 23% 1 8% 13 
UTR Cohort 6 14 100% 0 0% 0 0% 14 
TOTAL 65 82% 9 11% 5 6% 79 


Source: New Visions Data Warehouse 


ne The residency model for Cohort 6 varied somewhat from the model for Cohorts 1-5. 
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Completion and Attrition, by Subject 


Overall completion figures, across five cohorts, did not vary significantly by subject. Their numbers were small, but 
100% of the chemistry and earth science residents completed all program requirements. Biology had the next 
highest completion rate at 95%, followed by special education (82%) and English (81%); math residents had the 
lowest completion rates, at 76%. Rates for special education were six percentage points higher than mathematics, 
16 percentage points lower than combined science rates, and on par with English language arts. (See Figure 9.) 


Figure 9. Program Completion by Subject, Cohorts 2-6 
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Source: New Visions Data Warehouse 


Attrition figures show that most of the 14 special education residents who did not complete their residencies were 
counseled out, at somewhat higher rates than residents in other subjects. Of the 12 residents out-counseled from 
Cohorts 2-6, nine or 75% were special educators; one biology and two math residents were also out-counseled. 
Six percent of special education residents withdrew from the program, a rate lower than the rate of English (19%) 
and math (16%) residents; no science residents withdrew. (See Table 8.) 


Table 8. Residency Completion by Subject, Cohorts 2-6 


Total Enrolled | Completed Residency | Out-counseled | Withdrew 

Biology 22 21 95% 1 5% 0 0% 
Chemistry 3 3 100% 0 0% 0 0% 
Earth Science 1 1 100% 0 0% 0 0% 
English 36 29 81% 0 0% 7 19% 
Math 25 19 76% 2 8% 4 16% 
Special Education 79 65 82% 9 11% 5 6% 

TOTAL 166 138 83% 12 7% 16 10% 


Source: New Visions Data Warehouse 


Degree Completion and GPA, by UTR Cohort and Subject 


Hunter records indicate that almost all special education residents in Cohorts 3-5 (the cohorts for which data were 
available) completed their degrees on time, and at slightly higher rates than some of their cohort peers (See Table 
9). Records also show some slight variations in grade point averages—overall, math GPAs were a few tenths of a 
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grade point lower, and chemistry GPAs were the highest overall—but all residents posted GPAs of 3.7 or higher. 


The average across four cohorts for special educators was 3.85. (See Table 10.) 


Table 9. On-Time Degree Completion by Subject Area 


2010-2011 2011-2012 2012-2013 2013-2014 
Special Education 96% 100% 100% 
English (UTR) 100% 100% 100% 
Math (UTR/MASTER) 91% 83% 100% 100% 
Biology (UTR/MASTER) 80% 80% 50% 
Chemistry (UTR/MASTER) 100% 100% 100% 100% 
Earth Science (UTR/MASTER) 100% 100% 100% 

Source: Hunter College Data 
Table 10. GPA by Subject Area 

2010-2011 2011-2012 2012-2013 2013-2014 
Special Education (UTR) 3.82 3.83 3.84 3.92 
English (UTR) 3.89 3.79 3.93 
Math (UTR/MASTER) 3.74 3.69 3.89 3.66 
Biology (UTR/MASTER) 3.77 3.78 3.71 3.68 
Chemistry (UTR/MASTER) 3.94 3.8 3.93 
Earth Science (UTR/MASTER) 4.00 3.66 3.84 


Source: Hunter College Data 


GPAs: UTR vs. Other Hunter Special Education Graduate Students 


Special education residents trained through UTR performed similarly to students trained through Hunter’s traditional 


special education programs. The GPAs for the two groups were within a tenth of a point of each other in almost 


every year data were available. The widest difference was in 2013-2014, when UTR special education residents’ 


average GPAs were a tenth of a point higher than those earned by students in the traditional program, or 3.92 vs. 


3.82. (See Table 11. SE7-12 refers to other graduate students in Hunter’s Adolescent Special Education 


Table 11. Special Education Adolescent Generalist Degrees by Cohort 
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UTR Cohort 2 2010-2011 
SE7-12 | 2011-2012 


2011-2012 


SE7-12 | 2012-2013 


2012-2013 
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programs.) 
Program 
UTR Cohort 3 
UTR Cohort 4 
Source: Hunter College Data 
Placement 


Number 


ND 
3.82 
3.83 
3.83 
3.82 
3.84 
3.82 
3.92 


Teacher shortages was one of the reasons partners created UTR, so obtaining jobs is not a major obstacle for 


special education residency completers. Figure 10 below, which includes overall numbers, across Cohorts 2-6, of 
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residency completers who obtained jobs, shows that special education, along with math and science, all three 
areas where demand was highest, had the highest placement rates—all at 100%; English rates were still high, but 
13 percentage points below other subjects. 


Figure 10. Induction Year Hiring by Subject Cohorts 2 through 6 
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NEW VISIONS ASSESSMENT TOOLS AND RATINGS 


How Do UTR Special Educators Perform Compared to Peers? 


In 2011-2012, or UTR’s third year, and second year with special education residents, New Visions began 
developing a suite of assessment tools that allowed program staff to examine residents’ performance as well as the 
effectiveness of program elements and supports. This performance-based system incorporated the Charlotte 
Danielson Framework for Teaching, mandated in New York City and used by NYCDOE principals to assess 
teachers. New Visions staff set benchmarks appropriate for novice teachers, and along with mentors, observed 
residents four times a year, often debriefing together. Project staff also created an addendum to the Danielson tool 
for special education, which special education mentors helped draft and pilot, to capture what happens in ICT 


classrooms. 


Other tools in the New Visions assessment system included the Defense of Learning (DoL), successful completion 
of which was required for graduation, and lesson and unit design and professionalism rubrics. As the assessment 
system evolved, New Visions staff incorporated additional features into the DoL and lesson and unit design tools to 
ensure that they were instructional as well as evaluative—and that residents could use the feedback to revise their 
work. A mid-year DoL gave residents a scaled down, practice run to show how they used inquiry to support 
student learning; residents could also use initial feedback to submit lessons and unit designs for a second review. 


Defense of Learning 

The DoL was an opportunity for all residents to show how they had used the inquiry cycle and student data to 
diagnose students’ needs, design instruction, and assess learning. There were variations by cohort, but, on the 
whole, special educators performed on par with their peers, and most met or came close to meeting the 
benchmarks, with percentages increasing from Cohort 4 to Cohort 6. 
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= The percentage of special education residents meeting the defense of learning benchmark in Cohort 4 
significantly increased from 56% in January to 93% in June. 


= In Year 5, 89% of the Special Education residents met benchmarks for the first, mid-year DoL, and 100% 
did so for their final end-of-year DoL presentations. 


= 100% of special education residents in Cohort 6 met the benchmark both in January and in June. 


Lesson and Unit Design 

Benchmark data was sometimes uneven for special education residents, but show that, overall, the percentage of 
special education residents who met the lesson design benchmark as rated by their mentors increased from 
November to February. 


= The rates of Cohort 5 special education residents rose from 22% meeting the benchmark in November to 
75% in February. The percentage of special education residents in Cohort 6 meeting the benchmark 
started higher in November, at just over half or 54%, and grew to 64% in February. 


= Past reviews have shown that mentors’ ratings often tend to be higher than those posted by program 
staff, but in the case of the lesson design ratings for special educators, staff ratings topped mentors’. 


Professionalism 

Professionalism had been an implicit expectation from the start, but it became clear that residents needed explicit 
direction about school responsibilities such as participating in meetings, responding to feedback, and respecting 
school hierarchies and norms. New Visions introduced the professionalism rubric with Cohort 4. Data show that: 


= Special education residents across cohorts had high rates of professionalism at the various rating points — 


including those who eventually withdrew or were counseled out. 


= Of the 41 special education residents in Cohorts 4, 5, and 6 who received professionalism ratings early on in 
the school year, only two did not meet the benchmark. 


= 100% of the residents in the three cohorts met the professionalism benchmark by the end of the year. 


Danielson Framework 

UTR special educators’ Danielson performance indicated that residents, like their peers, were performing at levels 
appropriate for novice teachers—or between two and three (“developing” and “effective”) on a four-point scale 
ranging from “unsatisfactory” to “distinguished” or “exemplary.” Benchmark ratings show some variation across 
special education cohorts: 


= 100% of Special Education and English Residents in Cohorts 5 and 6 met both the December and May 
Danielson benchmarks, based on ratings by both mentors and New Visions program officers. 

= The percentage of special education residents in Cohort 4 who met the Danielson benchmarks varied 
widely by subject area and by rater (mentor vs. program officer). The percentage of Cohort 4 special 
education residents who met the Danielson benchmark as rated by their mentor declined from 39% in 
December to 28% in May. However, the percentage of Cohort 4 special education residents who met the 
benchmark as rated by their PO increased from 17% in December to 44% in May. 
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Reviews of special educators’ performance on each of the six individual domains (see Figure 11) also show scores 
generally comparable to ratings for residents in science, math, and English. Mean scores were lowest for Cohort 4— 
likely because the Danielson Framework was new and norming and inter-rater reliability mechanisms were not in 
place—but edged up for Cohort 5, and Cohort 6 had the highest average scores across all six domains. As noted 
earlier, mentors, typically, have scored residents a little higher than New Visions staff, especially in the earlier 
observations, but, by the end of the year, scores have usually converged. The trend was not as clear for special 
educators, and both mentors and program officers tended to score them a little higher than their peers. 


Like residents in other subject areas, special educators tended to score highest on Danielson domains related to 
classroom environment—engaging students, creating a culture of learning, and managing behavior. Conversely, 
residents have tended to score lowest on the domains that relate more directly to instruction—using assessments 
and questioning. Although differences were small, in Cohorts 5 and 6, this trend did not hold true for special 
educators, which may simply reflect differences in residents—or the evolution of the program and a deepening 
emphasis on diagnosing learners’ needs and more practice as special educators cycle through assessments and 
framing—or re-framing—questions for special needs students. For Cohorts 5 and 6, residents’ scores on Using 
Assessment and Questioning ticked up enough from previous cohorts to move them from “developing” status to 
“effective” teacher. Other differences in early and later cohorts may be due to the change in the model for Cohort 6, 
but the fact that these differences applied to Cohort 5 as well as Cohort 6 may also signal a maturing project. 


Figure 11. Special Education Residents' Danielson Averages, Cohorts 
3-6 
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Performance on Special Education Addendum 


In UTR’s fourth year (SY2012-13), and second year of using the Danielson Framework, New Visions created an 
addendum to better capture the developing skills and practice of special education residents. Program staff and 
mentors rated, over four successive observations, collaborative team teaching and differentiation and modifications 
for special education students in ICT classes, along with small-group instruction in resource rooms (program staff 
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only). Ratings show special education resident achieving average ratings just under 3 on the 4-point scale. We do 
not have the 2013-2014 or Cohort 5 scores, but Cohort 6 average scores were slightly higher, at 3.01. (See Figure 
12.) 


Figure 12. Special Education Residents' Danielson Addendum Averages, 
Cohorts 4 &6 
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How Do UTR Special Educators Assess Their Own Readiness? 


Residents vs. Peers 


Past reports have described two trends in residents’ self-assessments. First was an inverse relationship between 
residents’ self-assessments of particular skills and their Danielson scores. Across subjects, residents were typically 
less confident in areas where Danielson ratings showed strengths—in the classroom environment and management 
domains—and more confident in the instructional domains of assessment and questioning, where they received 
somewhat lower ratings. Second was a tendency for residents to assign higher ratings for how often they employ 
certain practices than for how confident they feel doing them. 


We observed this second trend in our summative analyses of survey data across cohorts and subjects, for which 
we created a set of scales based largely on the constructs that had framed our data collection: confidence, 
practice, engagement, learning environment, assessment, professional engagement, and support and school 
climate.2° We also created three sets of overall scales: 


= General instructional efficacy, with items related to teaching subject matter and improving student 


achievement; 


= Overall preparedness, which included confidence-level items about handling classroom management 
challenges, differentiating instruction, and using data to inform instruction. 


= Overall best practices, with items about the frequency with which teachers checked for understanding, 
asked thought-provoking questions, or created opportunities for students to work independently and 
collaborative. 


Survey responses overall indicated that residents felt generally confident, and that they engaged in practices 
encouraged in their training frequently. However, as Figure 13 shows, for all except the math residents, ratings for 


a All scales were based on four-point response options, and all examined for reliability, using Cronbach’s alpha, which showed 
that reliability for most scales was fairly high, within acceptable ranges (a ranged from .702 to .935). 
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what residents in Cohorts 1 through 5 did in the classroom—on the best practices scale—were a few tenths of a 
point higher than ratings for how prepared they felt doing them. The respective means for the special education 
group were 3.0, 3.0, and 3.3.2’ 


Figure 13. Scale Means, Special Ed vs. Peers 
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Some interesting differences emerged when we added Cohort 6 special educators’ data to the mix: the second 
trend continued, but with wider margins: the mean for overall best practices was 3.16, and for overall 
preparedness, 3.10, while the mean for general instructional efficacy edged down to 2.89. 


Some gaps also widened when we looked at the second trend. Cohort 6 scored highest on all the Danielson 
domains, but, Compared to other subjects and previous special education peers, they tended to rate their 
confidence and practice lower. As noted earlier, the Cohort 6 model departed some from the Cohort 2-5 model, 
with differences in the selection of residents, the support they received, and fact that these were part-time teachers 
of record, with more school responsibilities, and possibly students, classes, or co-teaching situations with more 
challenges, all of which could explain the slight downturns. 


We should also note that these means are not necessarily low: the Cohort 6 special educators’ ratings consistently 
hover around the “prepared” or “confident” mark, or the 3.0 level. The slight declines could reflect that phenomenon 
where higher performers are the more self-critical. It may also be that refinements in the special education program 
and the inquiry and formative assessment training, and more conversations during which residents unpacked their 
practice with mentors, led residents to assess their own skills differently. Figure 14 shows the trend lines across 
cohorts for the assessment scale. 


27 : : , / : ; 
Means are based on responses from Cohorts 2-5, since there were no special education residents in the first cohort. 
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Figure 14. Special Educators’ Self-Reported Confidence in Assessment Skills, Cohorts 
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Special Education Residents vs. Special Education Mentors 


A final set of comparisons may lend some credence to the fact that conversations and debriefings with mentors 
affected residents’ self-assessments—and point to other facets of the evolution of the UTR special education 
model. In preparing special educators for the range of challenges and school configurations they might encounter, 
according to New Visions staff, they have tried to “cover as much ground as possible,” and focused residency 
preparation increasingly “around collaboration.” When we reviewed special education residents’ and mentors’ 
survey responses, across cohorts, we discovered that for a subset of items, ratings converged with Cohort 6. What 
was, with the first cohorts, a margin of seven to nine tenths of a point on a four-point scale narrowed to four to six 
tenths. In some cases, the margins stayed the same but ratings for both groups ticked up or down in sync. 


What we saw in special education residents’ and mentors’ ratings on the professional engagement scale was that 
they converged in 2015, with Cohort 6. For the overall scale average, mentors’ ratings were just above residents’ 
(M=3.35 vs. 3.20); for the frequencies with which they reported collaborating with other teachers, means met at 
3.70, and for the frequency with which they participated in departmental or grade-level meetings, means for 
residents’ self-reports (VW=3.80) rose above mentors (MW=3.63). Means were lower, and the gap wider, for 
collaboration on school-wide decisions. (See Figure 15.) 
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Convergence may, again, reflect the fact that Cohort 6 residents were part-time teachers of record were more 


involved in the school community. It may also signal a maturing project, and the fact that the program succeeded in 


grooming proactive novice teachers ready to take on leadership roles. It may also suggest that the program pushed 


mentors’ development as well as residents. The next section explores each of these in the context of actual 


schools. 


Figure 15. Special Education Residents vs. Special Education Mentors, Cohorts 2-6 
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Section 3 


Impact on Schools 


The two previous sections suggest that the 65 UTR special 
educators who entered classrooms from 2011-2015 did so 
with skills and confidence. This section looks at how that 
readiness translated into practice. 


The findings are based on case studies in three schools 
that provided a range of special education configurations, 
percentages of students with IEPs, and numbers of UTR- 
trained special educators teaching there. Two big 
questions guided the data collection: 


" How do the practices of UTR-trained special 
educators differ from those of other special 
education teachers? 


# To what extent do UTR-trained special education 
teachers collaborate and team with students’ 
classroom teachers? 


As part of the data collection, we surveyed both special 
education and content teachers. Our survey sample was 
small so results are not definitive or statistically significant, 
but, along with other data collection, helped delineate the 
differences—and _ similarities—between UTR _ special 
educators and their peers. The differences were not in 
commitment to students: all the special educators we 
talked with expressed the same deep commitment to 
serving students, and all feel the strains of serving students 
whose needs and numbers continue to grow. There was 
also broad acceptance of the new city policies and a belief 


Compared to special educators from 
other programs, UTR-trained special 
educators say their programs better 
prepared them to assess & meet 
students’ needs. 


They also feel better prepared to teach in 
ICT settings. 


Like other special educators, they feel 
least prepared to write or monitor IEPs. 


A concentration of UTR special educators 
with an inquiry mindset & knowledge of 
how to operationalize inquiry & target 
students’ needs helps define & balance 
co-teaching. (School A) 


A “skillset around intervention” can have 
a broader effect, changing how resource 
rooms work & how sped students are 
assessed & perceived. (School A) 


Collaboration skills & a focus on 
relationships can help make team- 
teaching “seamless,” with no distinction 
between content & sped teachers. A 
focus on relationships can also remove 
labels for students & the stigma attached 
to special education. (School B) 


High numbers of students qualifying for 
special services requires “sped 
responsibilities” of all staff, & underscores 
the challenges of preparing special 
educators to meet diverse needs. 

(School C) 


that inclusion classrooms were better for students. Most special educators acknowledged that some content-area 


teachers are more collaborative than others; both groups also acknowledged the challenges of making an ICT 


classroom work. 


What differs is the skills UTR-trained special educators bring to schools and the way in which they undertake their 


roles, in team-taught classrooms and in the school as a whole. The case studies not only gave us insights into what 


UTR training looks like in practice, but also into how schools have managed change and how the UTR special 


education model has supported their efforts. 
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OVERALL FINDINGS 


Visits to schools, which included interviews with principals and special education coordinators, interviews or focus 


groups with teachers, and observations of ICT classrooms, were the main focus of our case data collection, but to 
gather additional and uniform data, we also invited special education and content area teachers to complete 
surveys. The surveys included a series of scaled questions about teachers’ views on their preparation or readiness, 
along with open-ended questions that asked teachers to characterize their special education program and describe 
its strengths, along with the challenges they faced in meeting the needs of special education students. 


Survey Respondent Profile 


As noted in the Methods section, the survey sample was small, at 30 teachers, and there was some imbalance in 


responses across sites. The special education (N=22 . 
P ( ) Figure 16. Survey Respondents by 
contingent represented most special educators are the three School (N=30) 
100% 
schools, but and content area (N=8) groups did not include : : 


the full contingent of either group at the three schools, and 67% 67% 
groups were not evenly represented; at School A, for 
example, no content area teachers submitted responses. 38% 33% 
Figure 16 shows the percentages of special education and | 0% 
content area teachers, by school, who made up the sample. —_— 
Special Education Content Area 
Teachers’ responses to background items showed that the 
majority taught 9h or 10th graders (80% and 67%, 
respectively); 40% taught 11th or 12th graders; 24 of the 30 


respondents taught more than one grade level. The survey 
Figure 17. Alt. Cert. 


Respondents by School (N=16) 
profession. The breakdown was fairly even: 16 (53%) of the 30 100% 


also asked teachers to indicate how they entered the 


respondents received their training in an _ alternative 


certification program, and 14 (47%), in a traditional program. 


Representation within groups was generally even as well: 10 50% 50% 


of the 22 special education respondents (46%) entered = § | 
teaching through traditional programs; 12 (54%), through 
alternative programs (6 through UTR and 6 through Teaching 
Fellows). Among the eight content area teachers, the Special Education Doment Aree 
breakdown was half and half, with three content area teachers mSchoolA MSchoolB WSchool C 
prepared through Teaching Fellows and one through Teach 


for America. Figure 17 shows the representation by group and by school. 


Although the sample limited what we could say definitively, teachers’ views on preparation created a valuable 
backdrop for the more descriptive case study findings. 
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Teachers’ Views on Preparation 


The survey included scaled items asking teachers how well their preparation programs prepared them to: 


= Create formative assessments for special education students 


= Plan instruction based on formative assessment and feedback from students 
= Meet students’ needs 


= — Tailor instruction to individual students’ needs 


= Co-teach or work in an ICT setting 


= Work in a resource setting 


= Develop, implement, or monitor IEP or 504 plans 


Special Educators Only 


Responses showed that, compared to other special educators, UTR-trained teachers felt that their program better 


prepared for every practice except developing IEP plans. All six of the UTR special educators, compared to 40 


percent of those prepared in traditional programs (4 out of 10) and a third of those (2 out of 6) prepared in other 


alternative certification programs reported that they felt “prepared” or “very well prepared” to use formative 


feedback (W=3.83 vs. 2.50 and 2.17, on a 4-point scale where 1=”not very well prepared” and 4="very well 


prepared”). The UTR group also felt better prepared to meet and tailor instruction to special education students’ 


needs, and create formative assessments for them. They—like their peers—felt less prepared to teach in an ICT or 


resource setting and develop IEPs or 504 plans. (See Figure 18 and Table 12, which shows percentages, by group, 


and UTR means in descending order.) 


Figure 18. Special Educators' Views on Preparation, by Program 
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Table 12. Case Study Special Educators’ View on Preparation (N=22) 


UTR (N=6) Teaching Fellows Traditional 
‘ (N=6) (N=10) 
Practices 9 ry, 
M 6 prepared/very M 0% M 0%, 
well prepared 
Plan instruction based on formative assessments & 
feedback from special ed students? 3.83 100% 2AL 383.3% 2.50 40% 
Tailor instruction to special ed students’ individual 
learning needs? 3.50 100% 2.17 33.3% 2.50 70% 
Meet the needs of special ed students? 3.33 88.3% 2.33 50% 3.10 90% 
Create formative assessments for special education 
Co-teach or work in an ICT setting? 2.67 66.7% 20 16.7% 1.90 30% 
Work in a resource setting? 2.67 50% 2.17 50% 2.70 70% 
9 ie) ie) 
Develop, implement, or monitor IEP or 504 plans? 1.88 33.3% oe 33.3% ony 50% 


Source: Case Study Teacher Survey 


Special Educators and Content Area Teachers 


We also examined special education and content area (general education) teachers’ responses to three items: how 
well their programs prepared them to teach in an ICT setting and meet the needs of special education students, 
and how well the program aligned to their current school’s approach. Figure 18 below shows the percentages of 
both groups who said they felt prepared or very prepared, or that their program was aligned or very well aligned; 
the special educators are divided by individual preparation program (UTR, Teaching Fellows, and traditional). 
Results indicated that UTR special educators and those prepared in traditional program felt their programs served 
them best, although the traditional group felt less prepared to teach in an ICT setting. Percentages of teachers from 
both these two groups indicated that their preparation program aligned with their school’s approach to special 
education were similar, and relatively high. (See Figure 19.) 


Figure 19. Special Educators' & Content Area Teachers' Views on Preparation 
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Case Studies 

The following three cases describe more about how special education and content area teachers’ work in ICT 
settings, and their own and school wide efforts to meet the needs of special education students. Although one 
criterion for selecting schools was their concentration of UTR-trained special educators, these are not necessarily 
comparative cases: School A, with the highest concentration, is an example of how a concentration of similarly 
trained UTR special educators can affect practice and mesh with school policies. School B is less an example of 
how fewer UTR special educators affect practice, than how they adapt and apply skills to existing school structures 
and policies. The case for School C is not as fully developed as the others because the UTR-trained special 
educator took on a new position in the school, then had to take a leave due to an unexpected illness. We have 
included the case because the school, with the highest economic index and number of students with disabilities, 
provides an example of the high-need environment all special educators encounter, and the approaches taken by 
special education and content area teachers to meet all students’ needs.”° 


28 The enrollment numbers and school demographic and performance data are based on 2015 figures, which may be slightly 
different from current figures. 
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BACKGROUND 


School A’s motto—“Supporting All Students, Each 
and Every Day’—echoes the name of the state 
credentialing exam and the commitment that special 
educators make when they sign up for the job. 


That’s no small task for a school that enrolls over 
8,300 students. As in many other NYC schools, that 
population is diverse: 34% of the students are Asian; 
30%, Black: 29%, Hispanic; and 4%, White. Ten 
percent of the students are English Language 
Learners, and 10% have disabilities that qualify them 
for special services. The school’s economic needs 
index stands at 59%. 


Unlike many large public schools, School A was not 
broken up into smaller schools as part of the NYC 
DOE’s 2002 reforms. Instead, the school was 
reconfigured into eight small learning communities 
(SLCs), with fewer than 500 students each. Designed 
around students’ interests and needs, the theme- 
based SLCs now include Health Sciences, Freshman 
Academy, Newcomers, Business/Technology, Pre- 
Med, Theatre, Public Service & Law, and Teachers of 
Tomorrow. Teacher leaders, called “directors,” run 
the SLCs, and each has its own counselor. 


According to the principal, these small schools create 
a “unique environment” in which students can pursue 
particular interests from a “menu of options for 
teens.” Students still see each other, at lunch, or 
during P.E. classes and extra-curricular activities, 
which include girls’ and boy’s basketball, bowling, 
and handball teams, as well as co-ed cricket and golf. 


The school can boast of some impressive sports 
victories and stats. But it’s other successes that they 
take special pride in, such as the Blue Ribbon School 
of Excellence award from the NY State Department of 
Education, or a graduation rate that is now 74%, 15 
points higher than the city-wide rate—and 15 points 
higher than it was when, along with their schools 


within a school initiative, they embarked on other big 
changes that affected special education. 


SPECIAL EDUCATION POLICIES 


Starting around 2005, the school had undergone 
what the principal, then the Assistant Principal, 
described as a “wall-to-wall transformation.” The 
changes were designed to address some critical 
academic performance shortcomings—including a 
50-60% graduation rate—and ensure that they were 
doing all they could to support a// students. 


It was because of these changes that the NYC DOE’s 
special education mandates did not require major 
overhauls at the school. According to the principal 
and the special education coordinator, changes were 
already underway. 


School A’s size allows it to offer a variety of classes, 
which special education teachers say is one of the 
key strengths of their program. It allows teachers to 
“cater to student needs.” The school offers not just 
variety, but also flexibility, latitude, and the “least 
restrictive environment for students.” The size of the 
school doesn’t limit interactions with _ individual 
students— sped teachers say the “student to teacher 
rapport” is strong. The teachers, they say, are 


ny ny 


“strong,” “caring,” “collaborative,” and support each 
other. The changes brought about by new policies, 
mandated by the NYCDOE and supported by school 
efforts, included new ways for special educators to 
work in the resource room. Team teaching was new 
for most teachers, so ICT classrooms evolved as 
changes were put in place, and both the principal, 
special education coordinator, and teachers say that 


UTR played a key role in that evolution. 


WHAT UTR SPECIAL EDUCATORS BRING 


Both the principal and special education coordinator 
believe the skills and dispositions that UTR special 
educators bring to ICT classrooms not only support 
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but often “inspire” changes. UTR-trained special 
educators, they say: 


> arrive familiar with “the latest methodology.” They 
know how, for example, to use and interpret the 
Danielson rubric. 


> have a solid “skill set around intervention.” The 
principal has seen former residents demonstrate 
these skills in Defense of Learning presentations. 


> are able to target specific skill needs, and “build” 
that skill. The Special Education coordinator has 
seen UTR-trained teachers do this with “a 
student they haven’t been able to support in 
other ways.” These kinds of skills, she says, 
“really benefit” the special education students. 


> are “more active” in ICT classes, including the 
four core Regents classes. Even the reluctant 
content area teachers are “sold” once a UTR- 
trained teacher has been able to “move” two or 
more students that the content area teacher 
“hasn't been able to move.” 


Content area teachers’ preparation may not have 
focused on instructional strategies that work with 
special education students, and, according to the 
principal, although they agreed with the philosophy 
behind inclusion, when schools shifted to ICT classes 
it was not as if they could “open up their suitcases” 
and pull out appropriate strategies. 


UTR teachers did, in a sense, “arrive with those 
suitcases.” They know the strategies and are ready to 
use them, a readiness that, according to the principal, 
distinguishes them not just from content area 
teachers, but also from other novice special 
educators with less training, and “less initiative,” and 
less certainty about “what the next step is.” 


Content area teachers who team with UTR special 
educators appreciate that they have an “inquiry 
mindset.” Compared to differently trained teachers, 


they are “better at driving instruction,” and “not timid 
at making decisions.” They are “willing not just to 
collaborate, but to try things, back it up.” 


A former UTR mentor notes similar skills, describing 
how a resident “brought new strategies from Hunter,” 
plus a certain exactitude about strategies: “everything 
has a reason,” ideas are “not flukes.” The special 
education coordinator sees a collective value in 
having multiple “UTRs,” who not only know how to 
reach this population,” and the knowledge to do, say, 
an “Item analysis of what kids need,” a preference for 
“using Google docs for planning and sharing,” but 
also a “willingness,” an “edge and desire.” 


WHAT UTR SPECIAL EDUCATORS SAY 


UTR-trained special educators say that assuming this 
role in their first induction year, after their residency, is 
not always easy: Even though their suitcases are 
stocked with a series of “do now’s, exit tickets, lots of 
informal assessment,” and they’ve “learned about 
groupings and learning style surveys,” and the value 
of “a good co-teaching relationship”—as a resident, 
right up until the school year starts, “you don’t really 
know what content area you'll be working in.” 


Asked, on the survey, about the school’s special 

education program and their own ICT classes, some 

residents described ICT classes and relationships that 

worked well: 

= Flexible-Equal among the two teachers [who] 
provide differentiated instruction, grouping, 
small group instruction, one-on-one instruction 
and maintenance of focus. 


. We co-design almost all assessments, for both 
special and general ed students 


» My co-taught classes have a_ beautifully 
balanced dynamic that includes shared 
responsibilities. 
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One UTR-trained special educator also described the full range of relationships: 

In the most effective class, we operate as a true partnership, both of us responsible for each 
student in the room. We co-plan for five minutes after each class about the next few days’ 
activities. We also do a lot of collaborating via email after hours. She writes the LPs and | 
differentiate them. She invites a constant dialogue about how to structure assessments and 
class activities. We trade lead-teaching responsibilities informally and often. We discuss specific 
students who we think require attention. We routinely engage in reflective practice. 

In the least effective classroom, the content teacher dictates what we're going to do and allows 
little room for discussion or, for that matter, differentiation. Co-planning with this teacher 
involves mostly just nodding my head. When this teacher complains about how much slower 
this class is than other classes with the same content, | explain that the other classes are not ICT 
classes. | am educating my colleague about special needs learners but it's slow-going and 
difficult to implement change in classroom procedure.... 

The other co-taught class is somewhere between these two extremes: the content teacher 
provides lesson plans and powerpoints, | provide differentiation and support for struggling 


students. Sometimes, we split the class into two groups and parallel teach using different 


methods of instruction. 


There are also logistical and operational challenges. 
some that are part of the job: it is, they say, difficult to 
manage high and low performing special education 
students in the same class. Time is always at a 
premium: there’s rarely enough “time in the day” to 
coordinate activities, develop and monitor IEPs, and 
“balance the needs of all students.” There is 
unanimous agreement that one of the biggest 
challenges is finding a common time to meet with the 
different content area teachers with whom they co- 
teach. How much planning and teaming go into 
lessons varies. With a large number of special 
education teachers working with many different 
content area teachers, there are “pockets” of 
teachers who are more collaborative than others. 


OBSERVING AN ICT CLASS 


During the observation, we saw _ collaboration 
between content area and special education teachers 
at work. We observed a Regents prep Global History 
class—with just 21 days left before a high-stakes 
exam often described as one of the hardest Regents. 
The content area teacher’s first question to the whole 
class reflected the hard task ahead, and the task at 
hand: “How can we review all the themes for our 
thematic essay exam?” Referring to previous 
assignments, he reminded students that the day 
would begin with a gallery walk of posters created by 
student teams for each theme, each one big, 
complex: Change & Ideas: Mao/Song; Scientific 


Revolution & French Revolution, Communism & 
Democracy. For the Do Now, students had to grapple 
with another question: Which problem faces many of 
the least developed nations today? The task, which 
provided fodder for much of the subsequent 
classroom discussion, was to annotate the seven 
posters using a template. 


As students started their walk, they talked, with both 
teachers reminding them of the task, the template. 
Students were then invited to contribute to a white 
board display and discussion of the Neolithic 
Revolution. Volunteers helped create a cause/effect 
diagram and worked on paraphrasing as_ both 
teachers reinforced key vocabulary words, such as 
“surplus” and “abundance,” and tackled tough 
questions: What is technology? What would qualify as 
technology in the Neolithic Era? What changes with 
land expansion? With technology + warfare? 

With about 10 minutes left in the class, students 
practiced for the exam with a multiple-choice hand- 
out. The special education teacher provided 
scaffolded practice for a group of five students, 
providing examples to discuss cause and effect; 
reviewing the multiple choice practice sheet; and 
using flash cards with images and key words— 
Industrial Revolution, inventions, urbanization, unions, 
and imperialism. 
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BROADER IMPACT 


Comments from the principal, cited earlier, about the 
fact that UTR special educators tend to be more 
comfortable than some novice teachers to take the 
initiative suggest that they are not just collaborative, 
but also proactive. One UTR = special educator 
described her own process, “| have to be assertive to 
implement a change but it can be done.” The special 
education coordinator described some results of UTR 
special educators’ proactive, leadership activities: 


> It has changed the way teachers work in the 
resource rooms, now supporting students with 
test preps, etc., and taking a more skills-based 
approach rather than memorizing. 


> Gen Ed has “learned, too,” and "added to their 
“tool belt” for ICT classes and even non-ICT 
classes. 


> More teachers are using assessment strategies 
that require multiple levels of thinking, rather than 
just recall. 


Even if all the practices of UTR special educators 
haven’t transferred to the larger school teams, there 
is, says the principal a “bigger benefit” in “how 
embedded they and their skills are in school culture” 
and the ways they “inspire” other teachers. UTR 
special educators have, he believes, 


> Helped increase the communication between 
teachers and the special education coordinator, 
including “early morning texting” and “sharing on 
Google.” 


> Supported residents in other cohorts. 
>» Changed the “perception of kids.” 
“Even some students,” he says, “have taught their 


gen ed teachers what their sped teachers taught 
them.” 
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BACKGROUND 


School B is a relatively new school that opened in 
2009, and, in 2012, expanded to a full high school 
with all four grades. It is based on a Career & 
Technical Education (CTE) model, created in 
response to a call to find new ways to prepare 
students for 21st Century careers. With a focus on 
sports business and enterprise, the school offers 
students not only a_ business’ entrepreneurship 
curriculum but also Industry partners and mentors.”° 


School B’s student population of around 400 
students, made up of students historically under- 
represented in the corporate world, do not typically 
have access to this kind of professional network: 59% 
are Hispanic; 35%, Black; and 2%, White, and 2%, 
Asian. Over a fifth, or 22% of the students qualify for 
special education services, and 7% are English 
Language Learners. The school’s Economic Need 
Index stands at 72%. 


Even though the school focuses on workplace skills 
and career success, it also places a good deal of 
emphasis on college readiness. According to the 
principal, the “high level’ curriculum and work 
expected of students is closer to “what kids are doing 
in college. When the school considers new programs 
or school reform initiatives, one of the first things he 
looks is “the strength of the curriculum.” 


BUILDING RELATIONSHIPS 


Over half of the students — 58% —who have IEPs, and 
thus qualify for special education services, spend over 
60% of the school week in ICT classes. The goal of 
those classes, and co-teaching model at School B is, 
according to the UTR-trained special education 
coordinator, is to make lesson planning and delivery 
“seamless.” That she says, isn’t so much a matter of 
skills, as  relationships—with teachers and with 
students. 


- School information from school website, https://www.nycboss.org. 


Those relationships are, she says, “different now,” as 
schools have adjusted to the NYC DOE’s inclusion 
requirements. It didn’t happen immediately, she 
says—and a content area teacher notes that some 
are still sometimes “uneven” —but both agree that the 
teams have “grown over time.” 


All this has, she believes, “revolutionized” instruction 
and classroom environments; content area teachers 
“feel the same way.” Almost all content-area courses 
are ICT paired. The content is different, but “routines” 
are similar: “We co-design almost all assessments, for 
both special and general education students.” The 
content-area teacher may create most of the class 
assessments, the special education teacher reviews 
or adapts, and they plan content delivery together. 


PREPARING STUDENTS 


On a broader level, teachers work together to blend 
the CTE and academic curriculum and find innovative 
ways to help special needs students who are 
struggling with the demands of both. They focus on 
those skills needed for the business workplace and 
college— especially communication skills that include 
presenting, synthesizing research, and _ writing 
persuasively. For teachers, it is, again, all about 
relationships and shared instructional routines such 
as a school-wide focus on literacy and Socratic 
seminars, where students research issues and mount 
evidence to make their case to teachers and peers. 


Relationships with students extend beyond the 
classroom, to tutoring in After-School and Saturday 
Academies. Tutoring has been re-branded—again to 
remove any stigma. They use cell phones apps to 
Google homework and check assignments, which 
has, according to the coordinator, resulted in a higher 
turn-in rate for homework. 


The sped coordinator has worked hard with the 
content area teachers to “remove the stigma” of 
special education students. Teachers assist students 
in Advisories, and steer them toward other support 
mechanisms, including internships in — sports, 
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journalism, and business; “Exposiums”— events used 
by high schools, universities, and organizations to 
expose students to new ideas and careers; and 
power lunches’ with business, media, and 
entrepreneurship guest speakers. 


CLASSROOM OBSERVATION 


There are three teachers in the Global History class: 
the content (social studies) and special education 
teacher, and a special education resident. The co- 
teachers work as a team, both circulating around the 
room, guiding students through an online curriculum, 
while the special education resident works with a 
small group of students at the back of the large room. 
These students are not necessarily those with IEPs, 
but those who, based on previous day’s activities and 
assignments, need extra support for the day’s lesson. 


The objective of that lesson—reviewed at the start of 
the class with students—is to analyze documents to 
identify origins of the world’s moral and legal codes 
and principles, dating back to the second century 
B.C. and the Code of Hammurabi, the King of 
Babylon. It’s a hefty subject, and the teachers guide 
students through skills and content together. The 
class starts with a Do Now, which takes about 7 
minutes, followed by a quick share out—a way to see 
if students grasp the concept. To provide some 
reinforcement and additional background for the 
day’s work, the social studies teacher takes students 
through a five-minute mini lesson. 


Much of the remaining class period is devoted to pair 
work, combined with activities projected on the white 
board, which shows passages, maps, and teachers’ 
comments. Locating Babylon and Mesopotamia as 
modern-day lrag, the special education teacher 
reminds students of the map skills they learned the 
previous day. The sped teacher walks students 
through the map. skills review, refreshing their 
memories on the title, key, compass rose, etc.” 


In keeping with the school-wide focus on literacy 
skills, as they walk around the room both teachers 
encourage students—individually and as a group—to 
use strategies such as annotating passages or 


highlighting key words, which can make answering 
the questions easier. 

Teachers direct students’ attention to the dictionary 
feature—another way to rely on tools. Next is a 
vocabulary review of terms: polytheism and 
monotheism. Both teachers recognize students who 
are “on point,” also reminding students about earning 
“above-and-beyond points” —credit awarded for extra 
effort. 


As students are working on activities, they get 
immediate, real-time feedback, in the form of Google 
comments. The pair work is broken up by a video 
and individual work. Students can also ask questions 
online—or do it the old-fashioned way, by raising their 
hands. They do both. The period ends with a wrap up 
and exit ticket work. The assignment is due—at 5pm, 
after many students are home or on their way 
somewhere else on a train. They turn in the 
assignment using their phones. 


At School B, it’s all about relationships. Special 
education and content area teachers work together 
to create seamless lessons that make it hard to tell 
which teacher is which. Students also move in and 
out of groups, willing to work with their peers in pairs, 
in small groups, or as a whole class— seamlessly. 
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BACKGROUND 


School C is one of five small schools housed on one 
of New York City’s most historic campuses. Built in 
1897, the castle-like structure, with its high ceilings 
and stained-glass windows, still stands. It was home 
to the first high school in the Bronx, named after one 
of the signers of the U.S. Constitution—and author of 
the Preamble—and also the alma mater of some well- 
known Americans, including former Secretary of State 
Colin Powell and playwright Clifford Odets. 


There is not just history here, but a history of reform: 
The original high school was the product of the NYC 
School Reform Act of 1896. Over a hundred years 
later, after the school, beset by challenges facing 
many large schools in the city’s high-poverty areas, 
had fallen into decline, the NYC Department of 
Education closed the original school as part of its 
2002 restructuring and downsizing reforms. What was 
once a single school became five smaller schools, 
including School C. 


The history of reform has continued, and School C 
can now add to the building’s storied past and 
famous graduates its recent selection as a 
Community School. These schools, a partnership 
between the NYC partners the Center for Supportive 
Schools, offer financial education, sports and arts 
education, computer training, | socio-emotional 
support—to serve all students, and serve the diverse 
needs of all students. 


Of the three case study sites, school A, with a student 
population of 471, has the economic needs index, at 
86%, and the highest percentages of English 
Language Learners (19%) and_ students’ with 
disabilities (25%). Close to three-fourths of the 
students are Hispanic (72%); 26%, Black; and 1%, 
Asian. The school’s Economic Need Index stands at 
85%. 


The school’s atmosphere reflects its student-centered 
focus: Samples of student work line hallways that 
feature original wood trim. Another prominent feature 
in the hallways is the principal, more often out of his 
office than in it. The family-like atmosphere extends 
beyond the school day: Students can get extra help 
on Saturday, and some participate in a mentoring 
program with professional adult volunteers. The 
principal, the signs and posters, the teachers—all 
point out that the school “offers students the 
opportunity to take charge of their lives.” 


SHARED RESPONSIBILITIES 


Implementing the school district's new special 
education policies did not change the collective sense 
of support for all students, but it did bring about new 
staff configurations. There are currently six special 
education teachers on the faculty, each of whom has 
some self-contained and some ICT classes. 
According to the special education coordinator, the 
ratio is about 15:1 in the self-contained classes, and 
12:1 in ICT classes, which typically have around 24 
students. The coordinator also stresses the whole 
school’s involvement in supporting all students: all 
staff members, she says, have what might be termed 
“sped responsibilities.” A School Implementation 
Team (SIT), made up of special education and content 
area teachers, guidance counselors, — and 
administrators, reviews both academic and _ social 
support, for transition, behavioral, guidance, and 
instruction. Parents, she says, are now more formally 
involved—staff members often talk with them on the 
phone, and parents “know sped teachers by their first 
names.” 


The transition included some efforts to “raise the 
consciousness of content area teachers,” but “good 


relationships” provided a foundation for that. In fact, 
says the coordinator, ICT collaboration—the “pre- 
planning, de-briefing, work during common planning 
times”—is “built on relationships.” Even those “on the 
sped team go to departmental meetings, often 
reporting back to the sped team.” 
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In response to survey items about how the special 
education program works, special education and 
content area teachers described the “dynamics” and 


” 


“division of responsibilities” as “good,” “collaborative”: 


= The ICT pairs work closely with the Gen ed 
teachers, aligning with what’s expected of all 
students. (Special Ed) 


= We have a good rapport with one another and 
the students. (Content) 


= Equal shares of the work. (Content) 


Teachers also described collaborative lesson 
development, which can vary some based on pairings 
and teacher preferences: 


= Special education teacher generally takes the 
lead; content-area teacher reviews. (Content) 


= The content area does the bulk of the planning. 
The special education teacher, when they get the 
opportunity, modifies the instruction. (Special Ed) 


= The Gen Ed teacher creates the work and the 
SPED teacher adds modifications. (Content) 


= | prepare the lessons with the science content to 
be taught and the Sped teacher will contribute by 
grouping students accordingly, taking daily 
attendance, reviewing Do Now and/or Summary 
with students as a whole class, perambulating as 
students work independently and/or 


collaboratively. (Content) 


PREPARATION AND PROBLEM-SOLVING 


The special education coordinator believes that it’s 
hard for new teachers to be, or feel, “fully prepared” 
because “the population is so diverse.” She notes that 
teachers’ preparation often tends to be theory-based, 
not quite in sync with the realities of serving a 
population for whom “literacy is a huge problem, but 
so are behavioral needs.” Teachers need to know 
how to help students with, for example, “basic 
measuring skills—how to use rulers, calculators” — 
and at the same time be able to diagnose needs, find 
“entry points” for students, develop “scaffolding for 
gaps in knowledge.” They need to know how to help 
students with “study skills, life skills”: how to complete 


a task, how to meet deadlines, how to do citations 
“All around,” she says, it’s a matter of coming up with 
“problem-solving strategies” and “creating flexible, 
responsive programs for students.” 


Especially with freshmen, the focus is on “transitions: 
getting them to the next level.” The special education 
coordinator sees 9th grade as a time to “get real.” 
The staff sees part of their role as “re-education about 
school is,” and helping students understand that 
“what you’ve done before won’t work.” It’s not just 
about messaging, but, from that point on, creating 
and sharing intervention plans, and “keeping very 
close track of these 120 students.” 


ICT TEAMS AND STUDENT PERFORMANCE 


One particular challenge for ICT teams is providing the 
support and monitoring that involves “crossover,” with 
division of labor switching depending on students’ 
needs. 


= Content-area teacher creates most class 
assessments; special education teacher reviews 
or adapts. 

= We co-design almost all assessments, for both 
special and general ed students. (Special Ed) 


According to special education and content area 
teachers (gathered for a focus group), the special 
education teacher might add to a test “with diagrams, 


pictures, 
ways to access them.” They may also reduce the 


not to simplify lessons, but give students 


number of questions on a worksheet or assessments 
for students with IEPs. 


To help students with the reading skills needed for 
various tasks and assessments, the school is planning 
to offer extra reading help three days a week, using 
the Wilson Reading System, a highly structured, 
remedial program where students are taught in small 
groups. 


At the same time, there is, say teachers and the 
special education coordinator, “constant monitoring, 
cycles of inquiry, baseline testing,” and ongoing 
efforts to design appropriate assessments for all 
students. These may range from school-wide units, to 
“less formalized” formative assessments. Teachers 
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often look for that “hinge point” using self-evaluation, 
such as the red/green/yellow strategy, where, as a 
class, students let the teacher know what they 
understand, what they don’t understand, or don’t 
quite get. 


For standardized tests, says the special education 
coordinator, they try to achieve a sort of “rhythm,” 
helping students with reading materials matched to 
levels, practice with writing, analysis, long-form 
essays—especially needed for high-stakes Regents 
exams in Global History and English Language Arts. 


As beneficial and necessary as the wide-ranging 
support for students is, this level of student support 
asks a lot of teachers. The issue is not that content 
area teachers and special education teachers are not 


collaborative, supportive of an “equitable division of 
duties,” but they are pulled in many _ different 
directions. A special education teacher is stretched, 
monitoring 40 kids who have individual educational 
plans, in addition to teaching a small group of special 
needs kids. Teachers may be asked to attend to other 
duties during co-teaching time—for “coverages, 
testing, meetings, PD's.” Again, teachers are 
committed to coordinating, co-planning, but often the 
collaboration, like the support for students, extends 
beyond the school day, with “email, telephone, 
Google sharing.” 


TAKEAWAYS ACROSS CASES 


The skills that UTR special educators develop during their training transfers to—and is evident in—their work 
as teachers of record. 


The clearest transfer of skills is in formative assessment activities, what the School A principal called a “skillset 
around intervention,” but other skills and dispositions— knowing when to advocate, when to pivot, when “rigid 
doesn’t work”—are also apparent. 


The roles they play in ICT classrooms may vary, depending on school size, structures, and previously defined 
team-teaching roles. School size, existing structures, and numbers of special education teachers and ICT 
teams vary between schools A and B, but UTR special educators have come to play instrumental roles in both. 
The division and equity of teaching duties may not be the same from teacher to teacher, but in both settings 
UTR special educators have helped steer changes, setting the standards for collaboration and defining the role 
of special education teachers. Team-teaching in School C, without a UTR presence, is still collaborative, but 
with more variation from team to team. 


What may be apparent in School A, with a larger corps of UTR special educators, is the important role they 
play in educating other teachers about the needs of learners with IEPs and strategies for supporting them. 


The emphasis on relationships with students may not only build students’ skills, confidence, trust, and interest 


in school: it may also help them stay in school. If UTR special educators’ efforts are also changing attitudes 
school wide; this, too, has the potential to improve special education students’ success and progress. 
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Section 4 


Impact on Students 


Findings thus far suggest that a sense of accountability for 
students’ learning is an important part of what UTR special 
educators bring to schools. All UTR-trained teachers are well 
versed in the inquiry cycle and formative assessment 
strategies, but, as the case study feedback made clear, 
special educators’ use of those skills—diagnosing needs, 
adapting team teachers’ lesson designs, targeting students or 
skills with on-the-spot scaffolding—can have a demonstrable 
impact on how ICT classrooms work. What UTR special 
educators bring to schools can also have a positive impact on 
how teachers, school-wide, meet—and view—special 
education students’ needs. 


The impact of these skills and practices on student 
performance is a little harder to confirm. Although many 
special education students sit for Regents exams and other 
end-of-course assessments, they may have waivers or 
accommodations, and, perhaps more important, the team- 
teaching model makes it hard to distinguish the impact of the 
special educator from that of the content area teacher. 


We have previously looked for other ways to explore student 
impact, examining, for example, whether having a UTR-trained 
content area teacher narrowed the achievement gap between 
students with IEPs and their peers. Although findings did not 
show a consistent, positive impact by subject, we did find 


11th & 12th grade special education 
students in schools with UTR sped 
teachers had higher attendance rates than 
sped students in schools with no UTR- 
trained teachers. Differences were 
statistically significant. 


Students in UTR schools also earned 
credits & passed Regents at higher levels; 
differences were not statistically 
significant. 


Data also indicates a positive impact on 
attendance, credits, & graduation rates for 
African-American & Hispanic students, for 
some comparisons, at significant levels. 


Some evidence indicated that 
concentrations of UTR-trained teacher 
mattered, but higher concentrations did 
not increase the effect. 


Gap analysis results were mixed: in half of 
the comparisons, having UTR teachers 
helped close the gap between special & 
general education students, with the 
narrowest gaps for credits earned. 


encouraging instances, in multiple subject areas, where IEP students’ Regent’s scores and grades neared those of 


general education students. For the supplemental research, we focused on schools with and without UTR-trained 


special education teachers, examining attendance, credit accumulation, and graduation rates —all institutional metrics 


that a shift toward a more nimble, granular approach to meeting special education students’ needs—and educating 


other teachers about meeting those needs—could affect. We also tested the concentration hypothesis guiding the 


case studies, to see if having more special educators, trained the same way, using similar strategies to meet 


students’ needs across content areas might collectively—and positively—affect these same outcomes. As a final 


metric, we again explored whether UTR-trained special educators helped schools close gaps between special 


education students and peers. 


12 


Samples and Research Design 


The findings discussed here are based on samples drawn from successive reviews of institutional data from close to 
2000 students and 70 NYC DOE schools in the New Visions affinity network. We assembled data from grades 11 
and 12 only because our focus was on students’ high school progress. The tables below include the numbers of 
students in the UTR and non-UTR designations, and in the UTR concentrations. 


We analyzed student- and school-level outcomes—attendance, credit accumulation rates, four- and five-year 
graduation rates, and the numbers of Regents, out of five, passed at 55 and 65—under two sets of conditions, 


1. Special education students only, UTR vs. Non-UTR. 


2. UTR concentration, special education and general education students, in schools with low, medium, and 
high numbers of UTR special educators, or no UTR-trained teachers. Our UTR concentration index used 
both the number and percentage of UTR-trained teachers at a school as well as the length of involvement. 


For the student-level analyses, students in the UTR group had to be attending a school at which there was at least 
one UTR-trained special education teacher during the 2014-2015 school year. Students in the non-UTR group had 
to be attending a school in which there were no UTR-trained teachers of any kind. Our rationale was that all UTR- 
trained teachers use similar strategies to diagnose and meet students’ needs, so a UTR-trained content area teacher 
could also have an effect on special education students. 


For the school-level analyses, schools in the UTR group had to have at least one UTR-trained special education 
teacher in 2014-2015 and at least 10 special education students in the specified grade level. To be designated as a 
non-UTR school, there could be no UTR-trained teachers of any kind (special ed or general ed) at any point over the 
past four years, and had to have at least 10 special education students in the specified grade level. 


For both conditions in the Gap Analyses, we examined school-level data for evidence that having UTR teachers on 
staff narrows the gap between special education students and their general education peers, based on attendance, 
credit accumulation, or graduation rates. 


The questions guiding the research were: 


«Do differences in teachers’ preparation emerge in special education students’ attendance, 
progress toward graduation (credit accumulation), or graduation rates? 


"Does a concentration of UTR-trained teachers in the school predict special education students’ 
success, as measured by these outcomes? 


UTR VS, NON-UTR 


Student-Level Comparisons 


Full Sample, All Special Education Students 


The first set of student-level analyses compared attendance, credit accumulation, and graduation rates among 
special education students only, in two conditions: those in schools with UTR-trained special education teachers, and 
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those in schools with no UTR-trained teachers (and no teachers trained through MASTER, an NSF-funded initiative 
using the New Visions—Hunter residency model to prepare STEM teachers). 


Though the differences were small, our comparisons indicated a positive impact. In each of the six comparisons, the 
scores were higher for the special education students in the UTR group (see Table 13). The differences were 
statistically significant (0<.01) for attendance data, for both 11th and 12th graders. 


= 141th grade special education students at schools with UTR-trained special educators, compared to 11th 
grade special education students at schools with no UTR-trained teachers, had higher attendance rates 
(86% vs. 81%) and accumulated more credits (85.79 vs. 35.04). 


= In 12th grade, special education students in the UTR condition again had higher attendance and earned 


more credits. 


= 12th graders in the UTR group passed 4.45 out of 5 Regents exams with a score of 55, compared to 4.22 
exams for the non-UTR group. Passing rates at the 65-point level ticked down, but the UTR group’s rates 
were again higher, with 3.19 vs. 2.96 exams passed, out of 5. 


Table 13 shows the results for each grade level and each measure for the UTR and non-UTR group. The blue 
shading indicates the higher UTR figures; the darker blue indicates which differences—listed in the next to last 
column—were statistically significant. The sample sizes for both groups and grades are in italics just below the table. 


Table 13. UTR- vs. Non-UTR: 
Special Education Students Only 


Grade Level Measure UTR Non-UTR Difference Significance 
Attendance Rate 0.81 0.05 p<.01 
11th Grade 
Credits Earned 35.79 35.04 0.75 p=.30 
Attendance Rate 0.80 0.07 p<.01 
Credits Earned 49.35 47.54 1.81 p=.01 
12th Grade 
Regents Passed out of 5 at 55 4.45 4.22 0.23 p=.05 
Regents Passed out of 5 at 65 3.19 2.96 0.23 p=.14 


UTR sample size: 374 (11th), 346 (12th); Non-UTR sample size: 223 (11th), 250 (12th) 


Source: New Visions Network School Databases 


Students of Color 


Large percentages of students at many NYCDOE schools are students of color, and may make up a disproportionate 
percentage of students eligible for special education services. Given that students of color likely make up a significant 
percentage of populations who could be affected by the efforts of UTR special educators, we disaggregated data by 
race or ethnicity. The results showed that: 


= Attendance rates were higher among African-American special education students taught by UTR-trained 
special education teachers, compared to rates among African-American special education students in 
schools with no UTR-trained teachers. The same was true for Hispanic special education students. 
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= Differences were statistically significant for African-American students in 11th grade, and for Hispanic 
students in 11th and 12th grade. 


= Both the African-American and Hispanic special education students also had higher Regents passing rates, 
though differences were not significant. 


= Results varied for credit accumulation. African-American 12th graders earned more credits than their peers, 
and differences were statistically significant. Hispanic 12th graders, however, earned fewer credits. 


Again, the blue shading indicates that rates or numbers of credits earned or Regents passed were higher for the UTR 
group, and the darker blue means that the differences between figures for the two groups were statistically 
significant. 


Table 14. UTR vs. Non-UTR 
AFRICAN AMERICAN Special Education Students 


Grade Level Measure UTR Non-UTR Difference Significance 

14th Gade Attendance Rate 0.85 0.80 0.05 p=.18 
Credits Earned 34.04 83.22 0.82 p=.53 
Attendance Rate 0.81 0.06 p<.01 

42th Grade Credits Earned 48.68 48.95 -0.27 p=.78 
Regents Passed out of 5 at 55 4.47 4.15 0.32 p=.08 
Regents Passed out of 5 at 65 2.99 2.69 0.30 p=.23 


UTR sample size: 125 (11th), 124 (12th); Non-UTR sample size: 81 (11th), 106 (12th), special education students only 


Table 15. UTR vs. Non-UTR 
HISPANIC Special Education Students 


Grade Level Measure UTR Non-UTR Difference Significance 

44th Grade Attendance Rate 0.80 0.07 p<.01 
Credits Earned 36.91 35.80 1.11 p=.24 
Attendance Rate 0.77 0.07 p<.01 

joa Grade Credits Earned 45.17 4.26 p<.01 
Regents Passed out of 5 at 55 4.40 4.12 0.28 p=.12 
Regents Passed out of 5 at 65 3.12 2.81 0.31 p=.19 


UTR sample size: 186 (11th), 171 (12th); Non-UTR sample size: 123 (11th), 112 (12th) 


Source: New Visions Network School Databases 


School-Level Comparisons 
Graduation Rates 


We used school aggregates to compare 4- and 5-year graduation rates for special education students in UTR vs. 
non-UTR schools. The rates were similar, but four-year rates among special education students in the non-UTR 
condition were higher; 5-year rates reversed, with the same margin of difference (0.05). 
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Table 16. UTR- vs. Non-UTR Schools: 
Special Education Students Only 


Measure UTR (n=18) Non-UTR (n=1 1) Difference Significance 


12th | 4-year Graduation Rate (class of 2015) 0.55 0.60 -0.05 p=.54 


Grade } 
5-year Graduation Rate (class of 2014) 0.63 0.05 p=.57 


Source: New Visions Network Schoo! Databases 


UTR CONCENTRATIONS 


Student-Level Comparisons 


We also examined student- and school-level data based on concentrations, exploring whether the numbers of UTR- 
trained special educators had an impact on student outcomes. We used the same metrics—attendance rates, 
credits earned, and Regents passed—and clustered schools with low, medium, and high concentrations of UTR 
special education teachers, and with no UTR-trained teachers, special education or otherwise. 


While the findings overall favored the UTR groups, they were mixed, and sometimes counter-intuitive. We found, for 
example, that in some cases the figures were higher for the low UTR concentration group than for the high 
concentration group, or that the non-UTR group was higher than the medium concentration group, which, in several 
cases, had the lowest figures. 


In Table 17 below, the darker blue indicates the highest attendance or credit accumulation figures, the medium blue 
the second highest, and the gray the lowest. The diagonal-patterned column shows the average figures across the 
UTR concentrations (low, medium, and high). The significance levels in the last column refer to the ANOVA with all 
four groups (non-UTR and low-, medium-, & high- concentrations). The p values in bold indicate that there are 
significant differences in the "set” rather than individual pairs. 


The results suggest that: 


= On average, attendance rates and credit accumulation rates were higher in UTR vs. non-UTR schools, in 
both 11th and 12th grades. 


o In 11th grade, attendance rates were higher in the low UTR concentration group than in the high 
concentration group; credit accumulation rates were reversed. For both, the medium concentration 
group had lower rates than the non-UTR group. 

o 12th grade attendance rates were similar across UTR groups, but lower for the non-UTR group. 
Figures for credits earned increased from low to medium to high concentrations, and the UTR average 
was a few percentage points higher than the figures for the non-UTR group. 


= 12th grade Regents passage figures were, again, higher for the UTR group, but, again, figures were lowest 
for the medium UTR concentration group. For both the 55- and 65-levels, number of Regents that the low 
UTR concentration group passed exceeded the number passed by the high UTR concentration group, 
though only by a few tenths of a percentage point. 
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Table 17. Students at Schools with Different Concentrations UTR-Trained Special Ed 


Teachers vs. Students at Schools with No UTR-Trained Teachers: 


Special Education Students Only 


at 


poke ' Medium High Non- *Sig. of ANOVA 
Concentration of , . ae 
: Concentration | Concentration UTR (not individual 
ellenacaas (n=95) (n=188) (N=223) comparisons) 
Teachers (n=91) ~ - ~ p 
11th | Attendance _ 
ee é 81 ‘ 
Grade | Rate 0.76 0.89 0.8 p< 
Credits 
35.04 p=.16 
Earned 
pow : Medium High Non- *Sig. of ANOVA 
Concentration of . : borer 
: Concentration | Concentration UTR (not individual 
UTE Special Ed (n=78) (n=190) (n=250) comparisons) 
Teachers (n=78) 7 7 ~ v 
Attendance 
0.85 0.87 0.80 p<.01 
Rate 
T2th | Credits 
Grade 47.37 49.11 47.54 p<.01 
Earned 
Regents 
Passed out 4.22 p=.01 
of 5 at 55 
Regents 
Passed out 2.96 p=.02 
of 5 at 65 


* The significance levels refer to the ANOVA with 4 groups (non-UTR & low-, medium-, & high- concentration). The p values in bold 
indicate that there are significant differences in the "set” rather than individual pairs. 


Source: New Visions Network Schoo! Databases 


Results of our analysis of 12th grade graduation rates were also inconclusive, with no significant differences. 


Compared to rates in schools with no UTR-trained teachers, four- and five-year rates were higher in schools with a 


low concentration of UTR special educators; five-year rates were higher for the high-concentration group. The 


medium concentration group, again, had lower rates than the other concentration group and the non-UTR group. 


Closing the Gaps 


Aggregating results to the school level and using this school-level concentration index as a predictor value, we 


examined the credit accumulation (by grade level) and graduation rates for IEP students. 


The findings of our gap analyses did not show marked or significant differences. Narrower differences between 


general education and special education students did suggest that: 


= UTR sped teachers may be helping close the gaps for credits earned—as other findings suggest as well— 


for both 11th and 12th graders. 


= The gaps also narrowed for 11th graders, for attendance, and for 12th grade 5-year graduation rates. 
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The blue shading in Table 18 indicates where difference favored the UTR group; the gray shading indicates a smaller 


gap—or difference between general education and special education performance —for the non-UTR group. The 


values in the next to the last column show the differences in gaps, which are slight. A minus indicates how much 


smaller the UTR gap between special education and general education students was; a plus, by how much it 


exceeded the non-UTR gap. 


Table 18. Gap Analysis between Special Education and General Education Students: 
UTR vs. Non-UTR 


UTR Non-UTR Diff in Gaps: 
GenEd SpEd Gop | GewEd: SpEa | C= UTR minus | Sig. 
(n=13) (neta) | COPED | inity) (nett) | Gen Ed |  Non-uTR 
vs. Sp Ed vs. Sp Ed 
41th Attendance Rate 0.89 0.86 0.90 0.85 0.05 -0.03 p=.37 
Grade | Credits Eamed 39.09 36.42 39.75 35.90 3.85 -1.18 p=.38 
Attendance Rate 0.89 0.85 0.83 0.81 0.02 +0.01 p=.66 
Credits Earned 52.46 48.78 3.68 51.91 47.78 4.13 -0.45 p=.77 
Regents Passed out 
4.95 4.37 0.58 4.83 4.33 0.50 +0.08 p=.71 
of 5 at 55 
12th 
Regents Passed out 
Grade - 4.78 2.98 1.80 4.59 3.14 1.45 +0.35 p=.29 
of 5 at 65 
4-year Graduation 
0.81 0.55 0.26 0.75 0.60 Ons +0.11 p=.11 
Rate (class of 2015) 
5-year Graduation 
0.84 0.68 Ome 0.81 0.63 0.18 -0.02 p=.76 
Rate (class of 2014) 
Source: New Visions Network School Databases 
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UTR equipped novice teachers to succeed and stay 
in the classroom. This final section examines 
retention among UTR special educators, compared 
to their UTR peers in other subject areas and to 
trends citywide. (The third chapter in the 
supplemental series takes a closer look at retention 
across all UTR subjects and cohorts.) 


One important trend in New York City’s public 
schools during the period just prior to UTR, and 
during its early years, was an increase in both the 
number and share of special education teachers, 
who made up only 17 percent of all teachers in 
2000-2001, but close to 25 percent a decade later. 
In addition, the share of teachers leaving the 
classroom early in their careers began to edge down. 
Of the nearly 9,000 teachers hired in 2000-2001, 41 
percent had left within three years. Of the 6,000 
teachers hired in 2008-2009, that figure dropped to 
30 percent.*° Recent research on retention also 
suggests that attrition rates nationwide were 
improving and perhaps not as severe as portrayed in 
earlier studies.°" 


Thus, as UTR partners crafted their residency model, 
they weren’t necessarily bucking the citywide 
trends—the numbers of special education teachers 
and retention rates were on the upswing—but UTR 


was above the curve. 


Section 5 
Retention 


93% retention rates among UTR special 
educators are a few percentage points higher 
than rates among peers, which stands at 91%. 


UTR-trained teachers in general are staying in 
the profession, but special educators are more 
likely to stay in initial placement schools. 


Citywide retention rates & rates among special 
educators ticked up over the period examined 
here, but UTR rates were still higher. 


Attrition among special educators in NYC high 
schools increased from teachers’ first & third 
years, while UTR rates held steady. 


Retention rates for UTR special educators 
exceed rates among teachers from alternative 
certification programs. Again, attrition among 
other early career teachers ticks up from year 
to year, but UTR rates hold steady. 


UTR special educators practice & foster skills & 
dispositions linked to retention: focusing on 
accountability & knowing what to do with data, 
collaborating with peers & being proactive — 
both can help special educators acclimate to 
schools & contribute to a positive climate 
school wide 


ae See NYC Independent Budget Office, “A Statistical Portrait of New York City’s Public School Teachers.” Available at: 


http://www. ibo.nyc.ny.us/iboreports/201 4teacherdemographics.pdf. 


ms Susan Headden, “Beginners in the Classroom: What the Changing Demographics in the Classroom Mean for Schools, Students, 
and Society,” Carnegie Foundation for the Advancement of Teaching (March 2014). See also, Emma Brown, “Far Fewer New 
Teachers Are Leaving the Profession than Previously Thought,” Washington Post, April 30, 2015. 
https://www.washingtonpost.com/news/local/wp/2015/04/30/study-new-teacher-attrition-is-lower-than-previously- 
thought/?utm_term=.6f3c36725378. 
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This section ends with a look at how UTR special educators’ training might have played a role in their high rates of 
retention. Included are residents’ survey responses about factors generally believed to be related to retention— 
administrative support, positive school climate, opportunities to collaborate, opportunities for leadership. °2 The 
section also revisits findings and insights from the case studies, discussing how the skills and malleability UTR special 
educators bring to schools might not only support their own integration and job satisfaction but also contribute to 
school-wide stability and retention. 


RETENTION RATES FOR UTR SPECIAL EDUCATORS 
How Do UTR Special Educators Compare to their UTR Peers? 


Retention Rates across Subject and Cohorts 


At 94 percent, retention rates for special educators are a few percentage points higher than the overall rate of 91%. 
Data for Cohorts 2-6 show that 60 of the 64 UTR-trained special educators are still teaching: two have left the 
classroom, and the status of two others is classified as “unknown.” Of the 60 still teaching, 58 or 97% are teaching in 
a NYC DOE public or New Visions Charter school. (See Table 19.)*° 


ne See, for example, Richard Ingersoll and Lisa Merrill, “Seven Trends: The Transformation of the Teaching Force: A CRPE Working 
Paper,” Consortium for Policy Research in Education (November 2012); Richard Ingersoll and Michael Strong, “The Impact of 
Induction and Mentoring Programs for Beginning Teachers: A Critical Review of the Research,” University of Pennsylvania, 
(February, 2004). See also, William H. Marinell and Vanessa M. Coca with the Research Alliance for New York City Schools, “Who 
Stays and Who Leaves: Findings from a Three-Part Study of Teacher Turnover in New York City Public Schools,” Steinhardt School 
of Culture, Education and Human Development, New York University, (March 2013); and David Perda, "Transitions into and out of 
Teaching: A Longitudinal Analysis of Early Career Teacher Turnover" (2013). Dissertations available from ProQuest. AAI3594959. 
http://repository.upenn.edu/dissertations/AAI3594959. 

88 This section shares data on 64 special educators and 73 other UTR teachers prepared in Cohorts 2-6. It uses a total of 64 UTR 
special educators, rather than the 65 used in other reporting, because one Cohort 3 resident did not leave the classroom because 
of dissatisfaction or other reasons linked to attrition, but was tragically killed during Hurricane Sandy. The overall total used here is 
therefore 137, not the total of 138 used earlier. 
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Table 19. Residents’ Current Teaching Status, Cohorts 2-6 


eosin Not a eg ee anaes Retention Unknown/ 
Teaching Rates No data 
Charter School Charter School 
UTR Cohort 2 34 5 25 3 82% 1 
Biology 8 1 6 1 88% 0 
English 9 2 ‘g 0 78% 0 
Math 4 1 3 ) 75% 0 
Special Education 13 1 9 2 85% 1 
UTR Cohort 3 29 3 25 1 90% 0 
Biology 7 2 5 0 71% 0 
Math 8 0 v4 1 100% 0 
Special Education 14 1 13 0 93% 0 
UTR Cohort 4 31 2 27 1 93% 1 
Biology 6 1 5 0 88% 0 
Chemistry 3 1 2 0 67% 0 
Earth Science 1 0 1 0 100% 0 
Math rg 6) 6 1 100% 0 
Special Education 14 0 13 0 93% 1 
UTR Cohort 5 24 0 22 2 100% 0 
English 15 ) 13 2 0 0 
Special Education 9 0 9 0 100% 0 
UTR Cohort 6 19 0 19 0 100% 0 
English 5 ) 5 0 100% 0 
Special Education 14 0 14 0 100% 0 
TOTAL 137 10 118 7 91% 2 


Source: New Visions Data Warehouse: Teacher Certification Program Retention Data Accessed March 23, 2017. 


Overall retention rates by subject (with sciences combined) show the highest rates among math graduates, followed 
closely by special education and English graduates; the combined rate for biology, chemistry, and earth science 
graduates is 80 percent. (See Figure 20.) 


Figure 20. UTR Retention Rates by Subject, Cohorts 2-6 


94% 93% 95% 
80% 


100% > 91% 
80% ~ 
60% ~ 
40% ~ 


20% ~ 


0% T T T T T 1 
Overall Special Ed English Science Math 


Source: New Visions Data Warehouse: Teacher Certification Program Retention Data Accessed March 23, 2017. 
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Mobility among UTR Teachers Who Stayed in the Classroom 


Records show that, of the 99 UTR graduates teaching in a New Visions network or New Visions charter school (those 
for whom we have mobility data), 64, or 65 percent are currently teaching at the same school where they began. 


Among the 35 remaining teachers, 33 changed school once; and two, twice.** 
Among the 64 graduates still teaching, stability rates are 15 percentage points higher for the 44 UTR special 
educators than for the 55 graduates of other subjects, or 73 percent vs. 58 percent. Records also show that: 


= 82 of the 44 special educators — 73% —are still teaching at the school where they started. Of the 12 teachers 


(27%) who changed schools, all did so only once. 


= Cohort 2 special educators had the highest rate of mobility, compared to their special education peers: 
44%, or 4 out of 9 UTR changed schools once. The rates for other cohorts are 23%, 31%, and 1%, for 
Cohorts 3, 4, and 5 respectively. Corresponding retention rates are: 56%, 77%, 69%, and 89%. 


= As noted in the Conclusions section of this chapter, some program revisions seem to have led to positive 
results, and less mobility after Cohort 2 may be explained by more emphasis in the training, for mentors 


and residents, on school integration of residents. 


= Teaching at the same schools, hired by residency schools, training sites. 


Table 20 shows mobility rates among UTR graduates, and mobility patterns, i.e., how many times teachers, by 
cohort and subject, changed schools. Table 21 shows how many years UTR graduates remained in the classroom 


before leaving. 


i Only one year of hiring data is currently available for Cohort 6. 
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Table 20. Mobility among UTR-Trained Teachers, Cohorts 2-6 


Residents Teaching in NYC DOE | Teaching at Same School | Changed Schools Once | Changed Schools Twice 
SnNY anal as oe Number Percent Number Percent Number Percent 
UTR Cohort 2 25 11 44% 12 48% 2 8% 
Biology 1 17% 4 67% 1 17% 
English 4 57% 2 29% 1 14% 
Math 1 33% 2 67% 
Special Education 9 5 56% 4 44% 0 0% 
UTR Cohort 3 25 15 60% 10 40% 0 0% 
Biology 5 1 20% 4 80% 
Math 7 4 57% 3 43% 
Special Education 13 10 77% 3 23% 0 0% 
UTR Cohort 4 27 20 74% 7 26% 0 0% 
Biology 2 40% 3 60% 
Chemistry 2 100% 
Earth Science 1 1 100% 
Math 6 6 100% 
Special Education 13 9 69% 4 31% 0 0% 
UTR Cohort 5 22 18 82% 4 18% 0 0% 
English 13 10 77% 3 23% 
Special Education 9 8 89% 1 11% 0 0% 
TOTAL, 4 Cohorts 99 64 65% 33 33% 2 2% 


Source: New Visions Data Warehouse: Teacher Certification Program Retention Data Accessed March 23, 2017. 


Mobility after Commitment Period 


UTR-trained teachers commit to four years of teaching in NYC DOE public schools or New Visions charter schools, 
and there are financial consequences for not meeting that commitment. Records show that, as of SY 2015-2016, 
10, or seven percent of the 137 UTR graduates trained in Cohorts 2-6 are not currently teaching. Of those, five left 
the classroom after four years. Of the other five, three left after one year, and two, after two years. 


Of the 10 UTR graduates who are no longer teaching, two are special education teachers. One left after four years, 
and one after two years. As noted above, data are missing for two other residents. 
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Table 21. Mobility among UTR-Trained Teachers Who Started Careers in NYC Schools, 
Cohorts 2-6 


Residents Who Began 


a acienis Teaching in NYC Schools, not 0 1 2 3 4 
teaching in NYC Schools as of | Years | Year | Years | Years | Years 
SY 2015-2016 
UTR Cohort 2 34 5 1 4 
Biology i, 
English 2 2 
Math 1 1 
Special Education 13 1 1 
UTR Cohort 3 29 3 1 1 1 1 
Biology 7 2 1 1 
Math 8 0 
Special Education 14 1 1 
UTR Cohort 4 31 2 1 1 
Biology 1 1 
Chemistry 1 1 
Earth Science 0 
Math 0 
Special Education 14 0 
UTR Cohort 5 24 0 
English 15 0 
Special Education 9 0 
UTR Cohort 6 19 0 
English 5 0 
Special Education 14 0 
TOTAL 137 10 3 2 5 


Source: New Visions Data Warehouse: Teacher Certification Program Retention Data Accessed March 23, 2017. 
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How Do UTR Special Educators Compare to Teachers Citywide? 


Hiring and Retention Trends Citywide, 2011-2014 
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A look at the numbers of teachers who joined and left the New York City Public Schools system as UTR’s special 


education residency program got underway, and began placing teachers, provides some context for UTR retention 
and turnover. According to New York City Independent Budget Office (IBO) data, over the decade prior to the 
beginning of UTR, the numbers of special education teachers increased, even as the numbers of teachers overall 


declined: in 2011-2012, there were 54,778 teachers in the system, almost 10,000 fewer than a decade earlier. In the 


same year, the number of special education teachers increased by around 5,000 teachers, from 13,183 to 18,595.°° 


= New York City Public School Indicators: Demographics, Resources, Outcomes. IBO. Available at: 
http://www. ibo.nyc.ny.us/iboreports/new-york-city-public-school-indicators-demographics-resources-outcomes-october-2015.pdf. 
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Breakdowns show some fluctuation in high school figures—which increased between 2011 and 2012, decreased 
between 2012 and 2013, then ticked up again in 2013-201 4.% 


A snapshot of attrition figures for SY 2014-2015 shows two trends in New York City schools: 


1. Attrition was lower among special educators hired in New York City’s public schools, compared to that among 
high school teachers overall. 


2. Attrition among both groups increased from year to year. For both, attrition rates more than double from one 
year to two years. From their first or induction year to their third year, special educators post a 17 percentage- 
point increase; high school teachers, a 22 percentage-point increase. 


Figure 21. Percentage of NYC Teachers (Hired in 2011-12, 
2012-2013, 2013-14) 
Who Had Left by 2014-15, after 1, 2, &3 Years 


34% 
28% 
25% 
17% 
12% 
8% 
2013-2014 (After 1 Year) 2012-2013 (After 2 Years) 2011-2012 (After 3 Years) 
New Special Education Teachers New High School Teachers 


Source: NYC IBO 


UTR vs. other NYC High-School Special Educators 


For a closer comparison, we also looked at retention rates among high school special educators in NYC schools, 
again taking a snapshot of SY 2014-2015. We found attrition rates rose for UTR special educators over their first 
three years—as they did for other special educators newly hired in NYC high schools—but at a slower pace. Figure 
22 shows the percentages of UTR and other special education teachers hired between SY 2011-12, SY 2012-2013, 
and SY 2013-2014 who had left NYC schools by SY 2014-2015, after their first, second, and third year of teaching. 


= Compared to 37% of their special education peers in NYC high schools, 23% of the UTR special educators 
who began teaching in 2011-12 (3 out of 13), were no longer teaching in NYC classrooms in SY 2014-15, 
after their third year. (This includes the one special educator no longer teaching and the 2 teaching 
elsewhere.) 


oe The NYC data reported in the following discussion and figures—including the comparisons to retention among UTR special 
educators—provides a snapshot for the last school year for which we have IBO data, or SY 2014-2015. In addition to providing 
additional UTR data, from Cohorts 1-6, the retention report provides further citywide retention data. 
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= After their second year, the margins widened: 25% of the NYC group had left, compared to 7% of the UTR 
group (1 teacher). 


= No UTR-trained special educators had left after their first year, compared to 12% of their NYC high school 
special education peers. 


Figure 22. Special Educators Hired from 2011-12 through 2013-14 Who 
Had Left NYC High Schools by 


SY 2014-15 
37% 
New HS Special Education 
Teachers NYCDOE 
23% 25% 
129 
7%, js BUTR Special Education 
i Residents 
2011-2012 2012-2013 2013-2014 


Source: NYC IBO; New Visions Teacher Data Warehouse 


Which Teachers Leave? 


UTR was one of several alternative certification programs created in response to teacher shortages and the need to 
staff all classrooms with effective teachers. A study based on data from the national Schools and Staffing Survey 
(SASS) shows that those programs significantly increased the proportions of alternatively certified teachers in the 
workforce. In a little over a decade, from 2000 to 2012, the share of teachers prepared in alternative certification 


programs nearly doubled, growing from 13 percent to 24 percent.” 


The study, like some others, also reports that teachers prepared in alternative certification programs are more likely to 
leave. Data from the IBO, which tracks NYC DOE teacher hiring and retention by preparation pathway, includes 
traditional programs, the NYC Teaching Fellows Program, and Teach for America. We added UTR rates, compared 
to general education figures and those for special education—taking a snapshot of retention rates for teachers hired 
in 2018, after one year (2014), and after two years (2015). (See Figures 23 and 24 below.) Comparisons showed that: 


= Among non-UTR teachers, retention was relatively high after one year, then began to drop off. Retention 
was in most cases a little higher for special education, and declines were a little sharper for general 
education. 


= The biggest drop after two years in the classroom was for Teach for America teachers, which fell from 89% 
to 38% for general ed teachers and from 86% to 49% for special ed—a not surprising drop because their 
commitment is two years. 


» UTR had higher retention and no declines. The UTR rate for general ed held steady at 87%; the UTR rate for 
special educators in their second year was 93% compared to 49% for Teach for America, 72% for Teaching 
Fellows, and 77% for traditional pathways. 


o Report available at https://nces.ed.gov/pubs2015/2015337.pdf. 
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Figure 23. General Education Hired in 2013, Still Teaching in 


2014 & 2015 
: 89% 87% 87% 
80% 76% 
69% 
58% 
38% 
Traditional Pathway NYC Teaching Fellows Teach for America UTR 


Source: NYC IBO; New Visions Teacher Data Warehouse 


Figure 24. Special Education Hired in 2013, Still Teaching in 


2014 & 2015 
aase 93% 93% 
9 
85% a: 80% ) 
72% 
49% 
Traditional Pathway NYC Teaching Fellows Teach for America UTR 
@ 2014 2015 


Source: NYC IBO; New Visions Teacher Data Warehouse 


WHY TEACHERS STAY OR LEAVE 


Researchers studying teacher retention generally agree on the factors that encourage teachers to stay in the 
classroom and in the profession: 1) a positive school climate, 2) opportunities for collaboration and shared decision- 
making, 3) support from administrators, and 4) available professional development and resources for teaching and 
learning. Some lists also include “high-stakes accountability systems.”°° Recent research also suggests that teachers 
with more training in teaching methods and pedagogy, especially training that has included having feedback on their 
teaching are less likely to leave, especially early on in their careers.°° In some instances, salary plays a role. 


sa Ingersoll, 2012. 


oe Education First, “Ensuring High-Quality Teacher Talent: How Strong, Bold Partnerships between School Districts and Teacher 
Preparation Programs are Transforming the Teacher Pipeline” (2016), available at http://education-rst.com/wp- 
content/uploads/2016/01/Ens. See also Richard Ingersoll, Lisa Merrill, and Henry May, “What Are the Effects of Teacher Education 
and Preparation on Beginning Teacher Attrition?” Philadelphia: Consortium for Policy Research in Education, University of 
Pennsylvania Research Report #RR-82 (2014)s. http:/Awww.cpre.org/sites/default/files/researchreport/201 8_prepeffects2014.pdf. 
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On surveys administered throughout UTR’s six cohorts we asked residents and mentors what might affect their 
decision to leave or stay in the classroom. Their feedback maps closely to research findings: both groups assigned 
the highest ratings to “Support from School Administrators,” “Positive School Climate,” “Time to Collaborate with 
Colleagues,” and “Ongoing, Relevant Professional Development.” (In general, teachers assigned lower ratings to 
“Higher Salaries,” although, interestingly, mentors’ ratings decreased over time, but residents’ ratings increased.) 


Findings shared in Section 2 of this chapter indicated that, as the UTR special education model matured, residents 
were more likely to attend departmental meetings and collaborate with colleagues, and rates rose in tandem with 
mentors, ticking up to levels just above those for subject-area residents. The professionalism scale that clustered 
these items reflected the same trends. This integration into school activities and culture—sometimes the result of 
proactive efforts—may have set the precedent for how UTR special educators viewed and even ensured their 
involvement in decision-making as they became teachers of record. These actions and dispositions may, in turn, 
have helped ensure retention. 


The case studies provided further evidence of the collaborative skills and dispositions UTR special educators bring to 
schools, not only contributing to shared decision-making but also proactively fostering it, and in the process helping 
schools manage the changes and challenges involved in shifting to an ICT model, making teams work, serving 
increasing numbers of special needs students—and adding to school stability. 


Although we did not specifically ask about accountability systems in reference to UTR special educators’ acclimation 
to and impact on schools, it may well be that their emphasis on accountability in the form of inquiry and formative 
assessment also contributes to school stability —not just because it can improve student performance, but because it 
can shift the accountability focus from outputs or summative assessment to inputs, or the formative work that targets 
students’ needs and improves their chance of success. A final factor, which the case study work hinted at, was the 
importance of administrator support in teacher retention: when principals like the one at the high-concentration 
school, which had hosted and hired multiple residents, seek out UTR teachers for the skills they bring to schools, 
they are likely to support those novice teachers, and advocate for school structures and practices that support 
collaboration and a meaningful role for special education teachers. 
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Conclusions and 
Recommendations 


This chapter drew on multiple data sources, including five years of survey responses from mentors and residents, 
resident background and performance data from Hunter and New Visions, interview and survey data from case 
studies, student performance data from schools that have hired UTR-trained special education and content-area 
teachers and schools that have not, and retention data from UTR teachers and teachers city-wide. 


Combined, the data indicate that UTR partners crafted an effective, versatile model, born out of a need not just to 
add teachers to the pipeline but also to ensure that they could meet the needs of students whose numbers were 
expanding and of schools shifting to new special education policies designed to meet those needs. 


As schools adjusted, so did UTR partners. As mentors and content-area teachers in host schools shifted to team 
teaching and took on new roles in inclusion classrooms, New Visions increasingly stressed collaboration and 
encouraged mentors, and residents, to add leadership to school roles. As the city adopted the Danielson Framework, 
New Visions adapted the rubric for their assessment suite, adding a special education addendum. Hunter revised 
courses and course maps to introduce residents to redefined disability categories and emerging best practices, and 
ensure that they could pass the EAS and other credentialing exams. They coordinated their practitioner-focused 
Learning Lab experiences and assessment training with New Visions’ Defense of Learning requirements, and 
together they helped residents combine the two with new EdTPA performance assessment for teachers. 


Sixty-five special educators over five cohorts met the requirements and challenges, with only a few concluding that 
either the program or profession wasn’t a good fit. Those who completed residencies maintained high GPAs, and, by 
the end of their clinical year, had passed that midway mark on the Danielson four-point scoring rubric, transitioning 
from “developing” to “effective” teachers. They continued to meet the challenges as teachers of record, exhibiting the 
adaptability that characterized the model itself. They didn’t just adapt to changes, but introduced changes, and 
helped schools manage change. And they stayed, with higher retention and lower mobility rates than their peers. 


The case studies confirmed that UTR special educators successfully transferred formative assessment and inquiry 
skills from training to practice—and these should remain at the core of special educators’ training: 


« — Askillset around intervention, with deliberate but nimble assessment of needs and targeted strategies; 
=» An emphasis on collaboration and shared decision-making, combined with 

= Leadership that is proactive and practical; 

= Efforts to play an advocacy role, and educate others about the learning needs of students with IEPs. 


Perhaps the bigger takeaway—and chief recommendation of this chapter—is that residents can and should be 
influential and adaptable at the same time. What this means for the UTR special education model is that the above 
practices should continue, with attention to the specific, often competing demands facing special education 
residents, and the likely variations in their clinical and placement schools. Two of the “Improvement Principles” laid 
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out by Anthony Bryk and co-authors in their work in improvement science are to make the work “problem-specific” 
and focus on “variability,” because change ideas may work in some places or under certain conditions, but not 
others.*° 


= Clarify expectations for new residents about the different school scenarios they might encounter. Consider 
including former residents, mentors, and content-area teachers in an orientation or residency essentials 


panel or open discussion, to give residents concrete examples of the “variability” in team-teaching. 


= Include short sessions to introduce residents to team-teaching configurations and strategies—including 
team IEP writing—to give them an overview without overwhelming them. Binders or an online go-to/contact 
site might also reassure novices that they don’t have to absorb everything at once. 


= To avoid focusing just on problems or likely challenges, balance sessions or conversations with examples of 
what worked, in what context: again, concrete examples can help residents and mentors anticipate what 
might work in their school structures, and what may require that they be adaptable before, being influential. 


= As residents begin their clinical experience, continue, in formal or informal interactions, to address specific 


needs or problems: 


o if a resident is placed in a content area where they don’t feel strong, set up a conversation (virtual, 
phone, face-to-face) with residents, mentors, or content-area team teachers who have faced similar 
challenges. 


o It may be possible to have mini-tutorials, where residents get a quick review of, for example, an Algebra 
| or Global History curriculum. 


o If there are specific student populations or skill needs, examples of assessment strategies that could 
relieve novice teachers of having to come up with new strategies out of whole cloth. The Hunter liaison 
could also provide examples of best practices around specific student needs, along with reassurance. 


o Similarly, shared lesson-plan adaptations, for specific content areas or skills, could help stock toolkits. 


= Consider similar activities for mentor training, where current or former mentors share mentoring or team- 
teaching experiences and real examples of challenges they’ve encountered and met. 


= Extend the focus on possible configurations and school structures during hiring sessions. Invite former 
special education residents and mentors to provide lists of questions for residents to ask during job fairs or 
hiring interviews. Providing examples of workable structures and strategies may also be helpful during 
induction support, for graduates experiencing challenges with team-teaching. 


=" Although it is valuable for special educators to feel like a community, and have training and support that 
acknowledges and addresses their unique needs, cross-subject, joint training and sharing sessions with 
content area residents that focus on teaming and variability could be valuable as well. Residents in other 
subject areas will be co-teaching with special educators, and they are all, in the end, committed to 
educating all students. 


a Anthony S. Bryk, Louis Gomez, Alicia Grunow, and Paul LeMahieu, Learning to Improve: How America’s Schools Can Get Better 
at Getting Better (Cambridge: Harvard Education Press, 2015), 12-14. 
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Chapter 3 


Training and Retaining 
(Good Teachers 


The retention study posed three key questions. The central one is whether UTR and MASTER teachers are staying in 
the classroom. The other two explore what aspects of the residency models affect retention, and whether they are 
making a difference. 


How do retention rates among UTR- Do the teachers trained in ar; 
& MASTER-trained teachers What resident characateristics or 


compare to rates among peers UTR and MASTER stay in preg amas i oe ea ue 
prepared through other pathways? NYC DOE classrooms? PSA NMI 


Organization of the Chapter 


The purpose of this chapter is to place these questions in context—of both program- and city-wide retention data, 
and in the larger landscape of staffing New York City’s high-needs schools. To this end, the chapter is organized into 
two parts: Part 1 discusses the retention status of UTR and MASTER graduates, examining trends by cohort and 
subject area and comparing resident retention rates with those of teachers trained through other preparation 
programs and teaching in New York City. Drawing on retention data, background data about residents, survey data 
from residents’ clinical year, and publicly available data about residents’ initial hiring schools, Part 2 examines the 
factors that might affect teacher retention and mobility, including personal characteristics, programmatic 
components, and school environments. A summary of findings follows the methods section, and a set of conclusions 
follows Part 2. 


METHODS 


For this study we used largely quantitative data, from multiple sources. To analyze the data, we used descriptive 


statistics, analysis of variance (ANOVA), chi-square analysis, and logistic regressions. Our data sources included: 


= New Visions Data Warehouse and Teacher Certification Program Retention Data. These two 
datasets provided detailed information on UTR and MASTER resident characteristics and retention status. 
To build this dataset, New Visions drew in large part on the NYC DOE Pedagogue records, which include 
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demographic and employment data about all NYC DOE staff. Where holes, gaps, or inconsistencies were 
evident, or in cases where the graduates were no longer teaching in the NYC DOE (and thus would not 
appear in the Pedagogue records), the program relied on induction coaching reports and resident self- 
reports provided in surveys or personal outreach to complete the dataset. 


Data from the Rockman et al administered UTR Teacher Surveys. Between the 2011-2012 and 
2014-2015 school years, UTR residents completed a survey about their preparation for and experiences 
during their first year teaching. Only items that were represented in all three waves of the survey (2011, 
2012, and 2018) were included in the scales to allow for a valid longitudinal comparison. (VOTE: Because 
we did not have MASTER data from 2011 and 2012, the survey analyses and Part 2 findings include only 
UTR residents.) 


We analyzed resident responses by their retention status as of the 2015-2016 school year. We also 
conducted a cross-walk between survey versions to ensure that only identical and/or very similar items were 
compared across waves. For our analysis, we grouped related survey items into thematic scales; our 
analysis also examines individual survey items that are particularly relevant to teachers’ decisions to stay in 
the field. A list of the survey items included in each scale, as well as the ratings and statistical significance 
for each scale and survey item is provided in the Appendix F. 


Independent Budget Office of the City of New York Public School Indicators Data and Retention 
Data Requests. Hiring and retention data for teachers in NYCDOE is publicly available through the 
Independent Budget Office (IBO), which tracks, among other trends, NYC DOE teacher hiring and retention 
data. Findings shared in this report are based on the IBO School Indicators Data Set. Additional retention 
data by preparation pathway was made available through a data request from Rockman specific to this 
study. 


New York City Department of Education School Quality Reports and School Performance 
Dashboard. This publicly available dataset provided detailed information on the characteristics of residents’ 
initial hiring schools. 
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SUMMARY OF FINDINGS 


Retention rates among both UTR and MASTER graduates are high, and mobility rates are low. 
Across six UTR cohorts and two MASTER cohorts, 89% of UTR residents and 94% of MASTER residents 
were still teaching, as of the 2015-2016 school year. The data indicate that although there is some mobility 


among UTR- and MASTER-trained teachers, overall stability is high. Graduates are not only staying but 
staying put, in encouraging numbers: During the 2015-2016 school year, 67%, or 84 of the 126 UTR and 
MASTER graduates still teaching in NYC DOE or New Visions Charter schools, were teaching at the same 
school where they were originally hired. 


Retention rates decline slightly the longer teachers are in the classroom, but, after six years, 
close to three-fourths of UTR’s first cohort are still teaching. Cohort 6 UTR and MASTER Cohort 2 
teachers had 100% retention rates in 2015-2016, the first year those residents served as teachers of 
record. UTR’s first cohort of teachers, who completed their residency in 2009-2010 and became teachers 
of record during the 2010-2011 school year, had a retention rate of 72%, in 2015-2016, after five years in 
the classroom. 


Aggregate data across programs and cohort years show that retention rates vary some by 
subject, as do the numbers of residents in each subject. Earth science had the highest retention rate 
with all four of the residents trained still in the classroom, though, when all science subjects are averaged, 
the overall retention rate in science is 87%, the same as the rate among English Language Arts residents. 
Special education and math had the second highest retention rates, at 94%. 


Ul 


R RETENTION VS. CITY-WIDE RATES AND RATES FOR PREPARATION PROGRAMS 


UTR graduates had a lower rate of attrition—by half—compared to new NYC DOE high school 


teachers. After three years in the classroom, 34% of NYC DOE high school teachers new to the classroom 
had left, compared to 15% of UTR-trained teachers. 


UTR-trained teachers also had lower rates of attrition, compared to peers prepared through 
other alternative certification pathways. After two years in the classroom, attrition rates among UTR 
teachers were lower—by half or more—than those among teachers prepared through the NYC Teaching 
Fellows Program, TeachNYC Select Recruits, and Teach for America. UTR also had less attrition than 
traditional programs, but rates were similar, with differences of a few percentage points (11% for UTR vs. 
16% for traditional programs). These comparisons are for overall NYC DOE attrition rates. Disaggregated 
data for secondary school teachers by preparation pathway was not available. 


Preparation program service requirements influence teacher retention, but have less of an effect 
in the UTR and MASTER programs. Of the UTR and MASTER graduates who are no longer teaching in 
NYC DOE or New Visions charter schools, 24% left after four or more years teaching, thereby fulfilling the 
program’s service requirement. In comparison, almost half, or 49% of all teachers trained through Teach for 


America exited after fulfilling the program’s two-year service commitment. 
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SCHOOL CHARACTERISTICS AS A FACTOR IN R 


CHER CHARACTERISTICS AS FACTORS IN RETENTION 


A resident’s highest level of education has a statistically significant correlation with their 
teaching status. Of the residents who entered the program with only a Bachelor’s degree, 90% were 
teaching as of the 2015-2016 school year, compared to 83% of residents with a Master’s degree, and 68% 
with a doctorate or equivalent degree. 


Background data for UTR and MASTER teachers also suggest that personal characteristics are 
associated with retention. Being Hispanic, female, having only a Bachelor's degree, or focusing on 
special education are associated with above average retention rates. There of course may be other 
extenuating factors, including those related to school context and sense of community. 


Correlation analyses based on survey data suggest that residents’ perceptions of efficacy and 
preparedness are related to the likelihood that they will remain in the classroom. At the end of their 
clinical year, residents complete a survey with questions about their confidence or preparedness. Analyses 
of scales based on clusters related to efficacy and instruction show that UTR and MASTER graduates who 
eventually left the classroom rated themselves lower compared to graduates who were still teaching. 


Residents’ confidence in handling classroom management challenges may be an important 
predictor of later retention. Residents’ ratings on the classroom management scale are a statistically 
significant factor when analyzing retention status. Our comparisons between those no longer teaching and 
those still teaching in NYC DOE schools showed that the largest gaps between the two groups, based on 
the residency-year self-assessments, were on the classroom management scale and the general classroom 
efficacy scales. 


NTION 


Data also indicate a connection between residents’ sense of the school climate—during their 
clinical year—and the likelihood that they will continue in the teaching profession. Again, analyses 
of ratings on the UTR teacher surveys, provided by residents reflecting on their clinical year, indicated that 
their host school experience and environment can play a role in future retention. Differences between those 
no longer teaching and those still teaching in NYC DOE schools, by current retention status, were 
statistically significant. 


The demographic composition of a UTR graduate’s initial hiring school influences retention and 
contradicts predominant trends. The percentage of white students in the school population and 
percentage of economic need at the initial hiring school are statistically significant indicators of UTR 
graduates’ teaching status. The greater the percentage of white students and the lower the economic need 
index, the more likely a graduate was to stop teaching within two years post residency. 


School safety is a statistically significant factor in retention among UTR-trained teachers. A one- 
unit increase in the Safety index of a resident’s initial hiring school results in 3.6 times the overall odds of 
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retention. This finding indicates that the safer a resident’s hiring school is rated to be, the more likely the 
resident is to continue teaching at that school.”” 


=" Mobility data suggest that school culture affects teachers’ decisions to switch schools. Factors 


such as school location, demographics, and student achievement ratings appear to play less of a role. 


“" The safety index is a composite of the five safety related questions from the annual school survey the NYC DOE administers to 
parents, students and teachers. Findings from the survey, including the safety index, are available through the NYC DOE School 
Quality Guide. 
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Section 7 
Resident Retention Trends 


A goal of both UTR and MASTER was to accelerate the learning curve for new teachers, making them more effective 
on their first day in the classroom and equipping them with the skills necessary to not only succeed but also stay in 
the classroom, contributing to school performance and stability. This chapter examines retention as well as mobility 
among the 155 UTR teachers and 32 teachers from MASTER’s first two cohorts. As a point of comparison, the 
discussion also includes citywide retention data from the city’s Independent Budget Office. 


INTERNAL COMPARISONS 
Retention Rates by Cohort 


Data thus far indicate that UTR and MASTER are achieving the goal of retaining teachers. Since the first cohort of 
UTR residents became teachers of record during the 2010-2011 school year, retention rates have been consistently 
high. Across the six UTR cohorts, 138 of the 155 residents, or 89%, were currently teaching as of the 2015-2016 
school year. Of those, 130 were teaching in NYC DOE or New Visions charter schools. (See Table 1.) 


Table 1. Residents' Current Teaching Status, UTR Cohorts 1-6 


Teaching in NYC Teaching outside 
: Not Retention 
Residents . or NV Charter NYC or NV Unknown 
Teaching Rates 
School Charter School 
UTR Cohort 1 18 2 12 1 72% 3 
UTR Cohort 2 34 5 25 3 82% 1 
UTR Cohort 3 29 3 25 1 90% ) 
UTR Cohort 4 31 2 27 1 90% 1 
UTR Cohort 5 24 0 22 2 100% 0 
UTR Cohort 6 19 0 19 0 100% 0 
Total 155 12 130 8 89% 5 


Source: New Visions Data Warehouse, Teacher Certification Program Retention Data, accessed March 23, 2017. 


MASTER rates look positive as well, overall just a few percentage points below the UTR rates for the two years when 
the programs overlapped. Among the 32 MASTER residents in Cohorts 1 and 2, 30 or 94%, were teaching as of the 
end of the 2015-2016 school year. As illustrated in Table 2, 28 residents were teaching in a NYC DOE or New 
Visions charter school. MASTER’s first cohort of teachers, who became teachers of record during the 2014-2015 
school year, had an 89% retention rate, and Cohort 2, a 100% retention rate, as of the 2015-2016 school year. 
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Table 2. Residents' Current Teaching Status, Master Cohorts 1 - 2 


Not Teaching inNYC Teaching outside Retention 
Residents Teachin DOE or NV NYC DOE or NV Rates Unknown 
3 Charter School Charter School 
MASTER Cohort 1 19 15 89% 
MASTER Cohort 2 13 13 100% 
Total 32 28 94% 


Source: New Visions Data Warehouse, Teacher Certification Program Retention Data, accessed March 23, 2017 


Retention rates across cohorts do decrease slightly as the number of years since residents entered the classroom 
increases, or the longer teachers are in the classroom. As shown in Figure 1 below, UTR Cohort 1, who completed 
their residency in the 2009-2010 school year and became teachers of record during SY2010-2011, had a retention 
rate of 72%, the lowest of the six UTR cohorts and two MASTER Cohorts. UTR Cohorts 5 and 6 and MASTER 
Cohort 2 had 100% retention rates in SY2015-2016, the first or second year that residents in those cohorts served 
as a teacher of record.*” 


Figure 1. Resident Retention Rates, SY2015-2016 


100% 100% 100% 
90% 92% 89% 
82% 
/ | 
UTR Cohort 1. UTR Cohort 2. UTR Cohort 3 UTR Cohort 4. UTR Cohort5 UTRCohort6 MASTER MASTER 
Cohort 1 Cohort 2 


Retention Rates by Subject Area 


Aggregating data across programs and cohort years showed some variation in retention rates by subject. As shown 
in Figure 2 below, chemistry had the lowest retention rate with, 75%, or nine of the 12 residents trained still teaching 
as of the 2015-2016 school year. Earth science had the highest retention rate, with all four of the residents trained 
either in UTR or MASTER still in the classroom. Special education (n=64) and math (n=33) had the second highest 
retention rates, all at 94%. 


“? In tables and figures, UTR results are in orange, MASTER results in blue, combined results in scarlet. 
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Figure 2. Retention Rates by Subject, 
UTR and MASTER 


100% 


94% 94% 
86% 87% 
i ; 
Biology (n=35) Chemistry (n=12) Earth Science (n=4) — English (n=39) Math (n=33) Special Education 


(n=64) 


It should be noted, as illustrated in Table 3, that there is wide variability in the size of cohorts across subject areas. 
Earth science, which has the overall highest retention rate, has the smallest cohort size with only one resident trained 
in each of the four cohorts. Chemistry, which has the second smallest cohort size with three residents trained each 
cohort year, has the lowest overall retention rate. In the case of Chemistry, one of the three residents trained is no 
longer teaching from three of the four cohorts. This brings the retention rate down to 67% in those three years. In the 
most recent Chemistry cohort, MASTER cohort 2, all residents are still teaching resulting in a 100% retention rate and 
bringing the overall rate for Chemistry to 75%. For subject areas with a small number of residents, one resident can 
have a significant impact on the reported retention rate. Conversely, for subject areas with a large number of 
residents, the impact of one resident on the retention rate for the group is less substantial. 


Table 3. Residents' Current Retention Rates by Cohort and Subject 


Biology Chemistry ain English Math pee 
UTR Cohort 1 75% (n=4) 67% (n=3) | 100% (n=1) 100% (n=10) N/A N/A 
UTR Cohort 2 88% (n=8) N/A N/A 78% (n=9) 75% (n=4) 85% (n=13) 
UTR Cohort 3 71% (n=7) N/A N/A N/A 100% (n=8) - 93% (n=14) 
UTR Cohort 4 83% (n=6) 67% (n=3) 100% (n=1) N/A 100% (n=7) 93% (n=14) 
UTR Cohort 5 N/A N/A N/A 100% (n=15) N/A 100% (n=9) 
UTR Cohort 6 N/A N/A N/A 100% (n=5) N/A 100% (n=14) 
MASTER Cohort 1 | 100% (n=6) | 67% (n=3) | 100% (n=1) N/A 89% (n=9) N/A 
MASTER Cohort 2 | 100% (n=4) | 100% (n=3) | 100% (n=1) N/A 100% (n=5) N/A 


Source: New Visions Data Warehouse, Teacher Certification Program Retention Data, accessed March 23, 2017 


Tables 4 and 5 below provide a detailed profile of UTR and MASTER residents’ teaching status by cohort year and 
subject area. 
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Table 4. Residents’ Current Teaching Status, UTR Cohorts 1-6 


Not sve DDE a Teaching outside Retention 
Residents Teaching NV Charter NYC DOE or NV Rates Unknown 
School Charter School 
UTR Cohort 1 18 2 12 1 72% 3 
Biology 1 1 75% ) 
Chemistry 0 ) 67% 1 
Earth Science 0 0 100% 0 
English 10 1 7 0 70% 2 
UTR Cohort 2 34 5 25 3 82% | 
Biology 8 1 1 88% 0 
English 2 0 78% 0 
Math 1 0 75% 0 
Special Education 13 1 2 85% 1 
UTR Cohort 3 29 3 25 1 90% 0 
Biology 7 2 5 0 71% 0 
Math 8 0 7 1 100% 0 
Special Education 14 1 13 0 93% 0 
UTR Cohort 4 31 2 27 1 90% 1 
Biology 1 0 83% 0 
Chemistry 1 ) 67% 0 
Earth Science 0 0 100% 0 
Math 0 1 100% 0 
Special Education 14 0 13 0 93% 1 
UTR Cohort 5 24 0) 22 2 100% 10) 
English 15 0 13 2 100% 0 
Special Education 9 0 9 0 100% 0 
UTR Cohort 6 19 0 19 10) 100% 10) 
English 5 0 5 0 100% 0 
Special Education 14 @) 14 @) 100% 0 
Total 155 12 130 8 89% 5 


Source: New Visions Data Warehouse Teacher Certification Program Retention Data, accessed March 23, 2017. 
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Table 5. Residents' Current Teaching Status, MASTER Cohorts 1-2 


Teaching in Teaching 
NYC DOE or outside NYC 
NV Charter DOE or NV Retention 
Residents Not Teaching School Charter School Rates Unknown 
MASTER Cohort 1 19 2 15 2 89% 0 
Biology 6 0 5 1 100% 0 
Chemistry 3 1 2 0 67% 0 
Earth Science 1 0 1 0 100% 0 
Math 9 1 7 1 89% 0 
MASTER Cohort 2 13 0 13 0 100% 0 
Biology 4 0 4 0 100% 0 
Chemistry 3 0 3 0 100% 0 
Earth Science 1 0 1 0 100% 0 
Math 5 e) 5 0 100% e) 
Total 32 2 28 2 94% 0 


Source: New Visions Data Warehouse, Teacher Certification Program Retention Data, accessed March 28, 2017. 


Of the 155 UTR residents and 32 MASTER residents, only eight (5%) and two (6%) respectively were teaching 
outside of the district (see Tables 4 and 5 above). (Anecdotally, we know that some of these teachers moved out of 
state rather than moving to schools in New York City suburbs or to private schools in NYC.) 


Resident Mobility between Schools 


In addition to retaining teachers in the classroom, a secondary component to the challenge of staffing high needs 
schools is teacher mobility among schools. A guiding tenant of the UTR and MASTER residency programs is to 
ensure that students who attend hard-to-staff schools have access to highly qualified teachers, year after year. But 
as teachers gain experience and confidence in their skills, they may be more selective in choosing where they want to 
teach,"’ and some data indicate that, as they progress through their teaching careers, teachers tend to move away 
from schools that serve large concentrations of low-performing students, low-income students, and/or students of 
color—i.e., higher-need schools.“ 


Mobility by Program and Years Teaching 


The data indicate that although there is some mobility among UTR- and MASTER-trained teachers, overall there is 
relatively high stability. Table 6 on the following page displays teacher mobility between schools by UTR and MASTER 
cohorts and subject areas. (Only one year of hiring data is available for UTR Cohort 6 and MASTER Cohort 2; they 
therefore have been omitted from this analysis.) During the 2015-2016 school year, 126 residents from UTR Cohorts 
1 through 5 and MASTER Cohort 1 were teaching in NYC DOE or New Visions Charter schools. Sixty-seven percent 
or 84 of those residents were teaching at the same school where they began their teaching career. Forty residents, or 
32%, had changed schools once, while only two residents had changed schools twice. Both residents who changed 


- “Recruiting Quality Teachers to Hard-to-Staff Schools,” Southeast Center for Teacher Quality. Retrieved from: 
http://Awww.nea.org/assets/docs/HE/mf_HuntHardtoStaff.pdf. 

4 Eileen Horng, “Poor Working Conditions Make Urban Schools Hard-to-Staff” (UC Berkeley: University of California All Campus 
Consortium on Research for Diversity, 2005). Retrieved from: http://escholarship.org/uc/item/0269b64 1. 
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schools twice were in UTR Cohort 2—one, a math resident and the other a biology resident. No residents had 
changed schools three or more times. 


The picture changes some as teachers’ time in the field grows. The percentage of teachers who switch schools 
generally increases, in part following the trend for teachers to change schools as they gain experience. To illustrate, 
of the residents in UTR Cohort 1 who became teachers of record during the 2010-2011 school year, 50% had 
changed schools by the 2015-2016 school year. In comparison, fewer, or 18% of residents in Cohort 5 and 7% of 
residents in MASTER Cohort 1, both entering the classroom in 2014-2015, changed schools the following year. 


Figure 3. Percent Of Residents Who Changed Schools at Least Once, by Cohort 


56% 
50% 
40% 
26% 
18% 
i 7 
—— 
UTR Cohort 1 UTR Cohort 2 UTR Cohort 3 UTR Cohort 4 UTR CohortS MASTER Cohort 1 


Source: New Visions Data Warehouse, Teacher Certification Program Retention Data, accessed March 28, 2017. 


Mobility by Subject 


Across subject areas, mobility was greatest for biology, with 57% of residents changing schools at least once by the 
2015-2016 school year. English/Language Arts had the second highest rate of mobility, at 48%. Chemistry had the 
lowest rate, at 17%, with only one of the six residents changing schools. Mobility by subject area is illustrated in 
Figure 4 below. 


Figure 4. Percentages Of Residents Who Changed Schools at Least Once, by 


Subject 
57% 
43% 
33% 
26% a8 
, ig 7 
Biology (n=23) Chemistry (n=6) “Earth Science (n= 3) English (n=21) Math (n=23) ” Special Education 


(n=43) 


Source: New Visions Data Warehouse, Teacher Certification Program Retention Data, accessed March 238, 2017. 
Tables 6 below details residents’ teaching status by cohort, subject, and mobility between schools. 
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Urban Teacher Residency Project 


Table 6. Resident Teaching Status, UTR Cohorts 1— 5 & Master Cohort 1 


Residents Teaching in NYC DOE or Teaching at % Teaching at Changed % Changed Changed % Changed Schools 
NV Charter School SY2015-2016 Same School Same School Schools Once Schools Once Schools Twice Twice 
UTR Cohort 1 12 6 50% 6 50% 0 0% 
Biology 2 1 50% 1 50% 
Chemistry 2 1 50% 1 50% 
Earth Science 1 0 0% 1 100% 
English 7 4 57% 3 43% 
UTR Cohort 2 25 11 44% 12 48% 2 8% 
Biology 6 1 17% 4 67% 1 17% 
English 7 4 57% 2 29% 1 14% 
Math 3 1 33% 2 67% 
Special Education 9 5 56% 4 44% 
UTR Cohort 3 25 15 60% 10 40% 0 0% 
Biology 5 1 20% 4 80% 
Math 7 4 57% 3 43% 
Special Education 13 10 77% 3 23% 
UTR Cohort 4 27 20 74% 7 26% 0 0% 
Biology 5 2 40% 3 60% 
Chemistry 2 2 100% 
Earth Science 1 1 100% 
Math 6 6 100% 
Special Education 13 9 69% 4 31% 
UTR Cohort 5 22 18 82% 4 18% 0 0% 
English 13 10 77% 3 23% 
Special Education 9 8 89% 1 11% 
MASTER Cohort 1 15 14 93% 1 7% 
Biology 5 5 100% 
Chemistry 2 2 100% 
Earth Science 1 1 100% 
Math 7 6 86% 1 14% 
Total 126 84 67% 40 32% 2 2% 


NOTE: Only one year of hiring data is currently available for UTR Cohort 6 and MASTER Cohort 2. 
Source: New Visions Data Warehouse, Teacher Certification Program Retention Data, accessed March 23, 2017. 
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Mobility Trends 


Our sample is too small and our data too limited to determine whether the UTR- and MASTER-trained teachers who do 
switch schools move in search of a different school environment or a school that is not high-need. We did, however, 
examine the publicly available data to see if any general mobility trends emerged. 


Complete School Quality Snapshot data was available for UTR and MASTER teachers’ initial hiring and change school 
for 33 of the 40 teachers who changed schools once. A review of this data did not indicate a uniform trend in mobility 
patterns. Averaging ratings across the seven School Quality Snapshot categories—rigorous instruction, collaborative 
teachers, supportive environment, effective school leadership, strong family and community ties, trust and student 
achievement—revealed no consistent mobility patterns. Eighteen teachers moved to a school with a higher average 
rating, 13 moved to schools with lower average ratings, and two teachers went to schools with the same average rating. 
There was also no clear trend for how student achievement might determine teachers’ mobility decisions. 


We also explored other data and variables that might indicate trends or shed light on why teachers change schools. 
Teachers were evenly divided in their choices about location, or whether to teach at a school in the same borough or a 
different borough, and there was also no one borough that teachers favored over others. Seventeen of the 33 residents 
who switched schools once continued to teach in the same borough. Of the 16 teachers who changed boroughs, five 
switched to Manhattan, five to Brooklyn, four to the Bronx, and two to Queens. 


Though examining the mobility patterns of teachers’ based on the student demographic composition of their previous 
and current schools was also inconclusive, reviews did indicate some differences. Figure 5 below indicates, in gray and 
orange bars, whether UTR- and MASTER-trained teachers moved to schools with a lower or greater percentage of 
students in each category. For example, more teachers went to schools with lower percentages of Asian students and 
Black students, than to schools with higher percentages: 19 vs. 14, and 21 vs. 12. By contrast, twice as many 
teachers— 22 vs. 11—moved to schools with more Hispanic students; the pattern was the same for White students, 
though figures were more similar. 


More teachers switched to schools with higher concentrations of ELL students than lower concentrations (20 vs. 13); that 


pattern was reversed for Special Education students (14 vs. 19). These counts may suggest that the percentage of ELL 
and Special Education students at a school influence a teacher’s decision about where to teach. 
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Figure 5. Count of Residents and Mobility by Student Demographics 


Lower Percentage ™ Greater Percentage 


Source: New York City Department of Education, 2015-2016 Schoo! Quality Snapshot 


Part 2 of this chapter examines factors that might influence teacher retention, chief among them a range of factors 
related to school culture. Our reviews of staff experience and student and teacher attendance— comparing statistics for 
teachers’ initial schools to those for the schools they switched to—suggested that these proxies for stability may be 
influential factors in teachers’ mobility decisions. We found that more teachers switched from schools with lower student 
and teacher attendance rates to schools with higher attendance rates. Specifically, 21 out of 31, or 68% of teachers 
switched to schools with higher rates of student attendance; even higher numbers, or 24 out of 31, or 77%, switched to 
schools with higher rates of teacher attendance. 


We also saw a somewhat counterintuitive trend related to another measure of school stability: While teachers appear to 
be moving in greater numbers to schools with higher rates of attendance, they are moving to schools where staff 
members have fewer years. Sixty five percent or, 20 of 31, teachers moved to a school with a principal with fewer years 
of experience than the principal at their previous school. Similarly, 21 of the 31 teachers, or 68%, moved to schools with 
a lower percentage of teachers who have at least three or more years of experience. This suggests that teachers may be 
looking for younger schools or schools that have new leadership and newer teaching staffs. 


Retention and Program Commitment Period 


UTR- and MASTER-trained teachers commit to four years of teaching in NYC DOE public schools or New Visions charter 
schools, and there are financial consequences for not meeting that commitment. Of the 187 residents in UTR Cohorts 1 
through 6 and MASTER Cohorts 1 and 2, 29 were not teaching in NYC DOE or New Visions Charter Schools, as of the 
2015-2016 school year. (Residents identified as not teaching, teaching in schools outside of NYC DOE or New Visions, 
and those whose status is unknown are counted as no longer teaching in NYC DOE or New Visions Charter Schools.) 
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Seven or 24% of these 29 residents left after four or more years teaching, thereby fulfilling their service requirement. The 


remaining 22 residents left before satisfying their service requirement. Six residents left prior to completing their initial year 


of service, nine left after their first year in the classroom, and six residents left after two years. Only one resident left after 


three years of service, one year short of satisfying the program’s teaching requirement. 


Table 7. Years of Post-Residency Service for Residents No Longer Teaching in NYC DOE 
or NV Charter Schools, UTR Cohorts 1-6 & MASTER Cohorts 1-2 


Number No Longer 


Number of Teaching in NYCDOE or 1 
Residents NV charter school as of Year 
SY 2015-2016 
UTR Cohort 1 18 6 2 
Biology 4 2 1 
Chemistry 3 1 
Earth Science 1 0 
English 10 3 1 
UTR Cohort 2 34 9 3 
Biology 8 2 
English 9 2 
Math 4 1 1 
Special Education 13 4 2 
UTR Cohort 3 29 4 2 
Biology 7 2 1 
Math 8 1 1 
Special Education 14 1 
UTR Cohort 4 31 4 i 
Biology 6 1 1 
Chemistry 3 1 
Earth Science 1 ) 
Math 7 1 
Special Education 14 1 
UTR Cohort 5 24 2 
English 15 2 
Special Education 9 0 
UTR Cohort 6 19 
English 5 
Special Education 14 
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Number No Longer 
Number of Teaching in NYCDOE or 0 1 2 3 4 5 
Residents NV charter school as of Years Year Years’ Years Years Years 
SY 2015-2016 
MASTER Cohort 1 19 4 3 1 
Biology 6 1 1 
Chemistry 3 1 1 
Earth Science 1 0 
Math 9 2 1 1 
MASTER Cohort 2 13 i?) 
Biology 4 0 
Chemistry 3 0 
Earth Science 1 ) 
Math 5 0 
Total 187 29 6 9 6 1 6 1 


Source: New Visions Data Warehouse, Teacher Certification Program Retention Data, accessed March 23, 2017. 


EXTERNAL COM 


Reviews of findings from the Independent Budget Office (IBO), which tracks NYC DOE teacher hiring and retention data, 
suggest that retention rates improved over the period when UTR partners joined other educators and policymakers to 
tackle the problem. The NYC DOE made changes to keep teachers in the classroom longer, with salary increases and 
bonuses for teachers who take on leadership positions and accept positions in high-need schools, but teacher turnover 
remained a problem.*° 


This section examines how retention rates among UTR-trained teachers compare to rates of other NYC DOE teachers, 
including those who entered the profession through different pathways. The comparison data comes from the IBO, and 
the ways in which the IBO reports figures to a large extent determined how UTR and MASTER data are parsed and 
reported.*® 


UTR Retention Rates vs. Rates Citywide 


Findings are again positive, and favor UTR. Data from the IBO and UTR program data show that UTR residents leave 
NYC schools at lower rates than other new high school teachers hired by NYC DOE in the same year, by fairly large 
margins. As illustrated in Figure 6 below, 34% of the new high school teachers hired by NYC DOE in 2011-2012 had left 
by the 2014-2015 school year, or after three years, compared to only 15% of the UTR trained residents. UTR residents 
hired in 2012-2013 and in 2013-2014 continued the trend. Although attrition in both groups increased each year, fewer 
UTR teachers than other high school teachers left the classroom, with 10% vs. 28% vs. leaving after two years, and 3% 
vs. 12% leaving after just one year. 


oo Geoff Decker, “Five Data Points that Illustrate Challenges New York City Schools Face,” Chalkbeat, Oct. 6, 2015. Available at 
Cityhttps:/Awww.chalkbeat.org/posts/ny/201 5/1 0/06/five-data-points-that-illustrate-challenges-new-york-city-schools-face/ 


= Because of the time periods for which NYC DOE/IBO data are available, MASTER graduates are not included in this discussion. 
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Figure 6. High School Teachers Hired from 2011-12 through 2013-14, Still Teaching 


In SY2014-15 
34% 
28% 
NYCDOE New HS Teachers 
15% 
10% 12% UTR Residents (Cohorts 2,3, and 4) 

i ~ 

2011-2012 2012-2013 2013-2014 


Source: IBO teacher retention data. 


Retention Rates by Teacher Preparation Pathway 


The IBO tracks NYC DOE teacher hiring and retention by preparation pathway, including traditional programs, the NYC 
Teaching Fellows Program, TeachNYC Select Recruits, and Teach for America. Comparisons of the IBO data and UTR 
program data show that UTR-trained teachers had lower rates of attrition from NYC public schools after one and two 
years of teaching, compared to their peers prepared through other pathways. 


Attrition rates after the first year of teaching varied across preparation pathway, with a high of 18% leaving from the 
Teach NYC Select Recruits preparation pathway, compared to a low of 7% for UTR-trained teachers. The second lowest 
attrition rate, after UTR, is the traditional pathway; the IBO data show that 9% of teachers prepared through traditional 
programs exited NYC DOE schools after one year. 


Attrition rates increase across all preparation pathways after teachers’ second year in the classroom, but UTR attrition 
rose by fewer percentage points. The UTR attrition rate increased from 7% after the first year to 11% after the second. 
Teach for America had the largest increase, with attrition rates rising sharply from 10% leaving NYC traditional schools 
after one year in the classroom to 49% leaving after two years. This finding suggests that there is a direct link between 
Teach for America’s two-year service commitment and the almost 50% attrition rate for teachers prepared under this 
pathway after their second year. Again, attrition among teachers prepared in traditional programs were the second 
lowest behind UTR’s rates. (See Figure 7.) 
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Figure 7. Teachers Hired In 2012-13, Percentage Who Left NYC's Public Schools 
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Source: IBO teacher retention data. 
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section 2 
Factors Influencing Retention 


The research literature points to a consistent set of factors related to teacher retention, most centered on working 
conditions or job satisfaction: teachers benefit from, and tend to remain in schools where there is administrative support, 
a positive school climate, opportunities to collaborate, and opportunities for leadership.’’ Some research also indicates 
that teachers’ sense of success in the classroom—their sense of safety, their confidence in classroom management and 
ability to meet students’ learning needs—can also affect their decision to stay or leave.’” Program records allowed us to 
track retention and mobility trends for UTR and MASTER graduates, but except for interviews with a sample of MASTER 
residents in their induction years, we did not follow residents into placement schools or gather longer-term feedback 
about job satisfaction or sense of success. What we did have were four other sets of pertinent data. 


= The first data source was background data we drew on to examine links between personal characteristics — gender, 
ethnicity, education level—and mobility and retention. 


= The second source was feedback from residents at the end of their clinical year, as they prepared to enter NYC 
DOE classrooms as full-time teachers of record. These data allowed us to examine links between residents’ sense 
of efficacy and preparation, and mobility and retention. 


= We also had survey feedback about residency-year school environments—or residents’ sense of community and 
support throughout their clinical years— which, similarly, allowed us to explore whether the environment in which a 
resident gained clinical experience had any bearing on subsequent mobility and retention. 


= The fourth data set is drawn from publicly available information about residents’ placement schools. We assembled 
a set of school factors that included school demographics; economic need index; student achievement levels; and 
ratings for safety, support, collaboration, and family and community ties. 


As noted earlier, UTR and MASTER preparation is designed to build residents’ instructional, management, and 
collaboration skills, help them acclimate to urban environments, and give them the fortitude to weather environments that 
fall short of expectations or differ from residency-year schools. This section speaks to the success of that training, or how 
the recruitment, training, and support mechanisms in the residency model might anticipate and mitigate attrition, 
especially attrition in urban schools. 


“" Richard Ingersoll, Lisa Merrill, and Henry May, “What Are the Effects of Teacher Education and Preparation on Beginning Teacher 
Attrition?” (Philadelphia: Consortium for Policy Research in Education, University of Pennsylvania, 2014), Research Report #RR-82. 
Available at: http:/Awww.cpre.org/sites/default/files/researchreport/2018_prepeffects201 4. pdf. 


“8 Karen J. DeAngeles, Andrew F. Wall, and Jing Che, “The Impact of Preservice Preparation and Early Career Support on Novice 
Teachers’ Career Intentions and Decisions,” Journal of Teacher Education 64, no. 4 (2013): 338-55. 
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SUMMARY OF TEACHING STATUS 


The analysis in this section excludes any residents with incomplete retention data and those whose teaching status was 


unknown as of the 2015-2016 school year. Therefore, the counts of residents here may differ from those cited earlier in 
the chapter. Complete retention data was available for 182 residents trained in UTR Cohorts 1-6 and MASTER Cohorts 
1 and 2. Table 8 below provides a breakdown of counts by cohort included in this analysis. 


Table 8. Residents with Complete Retention Data, 


by Cohort 

N % 
MASTER Cohort 1 19 10.4 
MASTER Cohort 2 13 7A 
UTR Cohort 1 15 8.2 
UTR Cohort 2 34 18.7 
UTR Cohort 3 28 15.4 
UTR Cohort 4 30 16.5 
UTR Cohort 5 24 13.2 
UTR Cohort 6 19 10.4 
Total 182 100 


Of the UTR and MASTER graduates included in the analysis, 86.8% were teaching in NYC DOE or NV Charter Schools 
as of the 2015-2016 school year. Ten residents, or 5.5%, were teaching outside of NYC DOE or NV Charter Schools, 
and 14, or 7.7%, were no longer teaching. The group of 14 graduates who are no longer teaching was divided based on 
the length of their tenure in the classroom. Five teachers stopped after one year of teaching, and another five after three 
or more years. For those no longer teaching, the mean number of years in the classroom was 2.1 years. (See Figure 8.) 


Figure 8. Residency Graduates No Longer Teaching 


Stopped after residency year (n=2 14% 


Stopped after 1 year of teaching (n=5) sex 


Stopped after 2 years of teaching (n=2 14% 


Stopped after 3 or more years of teaching (n=5) [I scx 


Source: New Visions Data Warehouse, Teacher Certification Program Retention Data, accessed March 28, 2017. 


Of the 10 residents who were teaching outside of the NYC DOE as of the 2015-2016 school year, four exited the NYC 
DOE after their residency year, three left after one year of teaching, two left after two years, and only one left after three 
years. This finding suggests that the initial years are critical, and that the longer they teach the less likely teachers are to 
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exit from the NYC DOE. The average number of years taught in NYC DOE before leaving to teach elsewhere is one. (See 


Figure 9.) 


Figure 9. Residency Graduates Teaching Outside NYC DOE 


Left NYC after residency year 


Left NYC after 1 year of teaching 


Left NYC after 2 years of teaching 


Left NYC after 3 or more years of teaching 


40% 


Source: New Visions Data Warehouse, Teacher Certification Program Retention Data, accessed March 23, 2017. 


For the combined group of 24 teachers who are either no longer teaching or who have left the NYC DOE to teach 


elsewhere, the average time spent teaching in NYC DOE schools was 1.6 years. A third of those who stopped teaching 
or left the NYC DOE did so after their first year of teaching. (See Figure 10.) 


Figure 10. Residency Graduates No Longer Teaching or Left NYC DOE 
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Source: New Visions Data Warehouse, Teacher Certification Program Retention Data, accessed March 23, 2017. 


For this analysis, UTR residents in Cohorts 1 through 6 and MASTER Cohorts 1 and 2 with complete retention data were 


grouped into the following four categories—currently teaching inside NYC, currently teaching outside NYC, stopped 


teaching within two years post residency, and stopped teaching three or more years post-residency. The number of 


residents in each of these categories is summarized in Figure 11. 
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Figure 11. Teaching Status Summary 
Currently teaching inside NYC (n=158) [J s7% 


Currently teaching outside NYC (n=10) [Eg] 6% 


Stopped teaching within 2 yrs post-residency (n=9 al 5% 


Stopped teaching 3 or more years post-residency (n=5 | 38% 


Source: New Visions Data Warehouse, Teacher Certification Program Retention Data, accessed March 23, 2017. 


RESIDENCY CHARACTERISTICS AND RETENTION 


We explored a number of programmatic, school, and teacher characteristics that might play a role in UTR and MASTER 


teachers’ acclimation to schools as teachers of record, and thus in retention. Using program data and survey data 
collected during residents’ clinical year, we selected and examined the impact of 22 different program and teacher 
characteristics on a resident’s retention and teaching status (as of the 2015-2016 school year). We used three different 
analysis methods to measure impact: Chi Square, ANOVA, and Logistical Regression. Findings from these analyses are 
organized according to characteristic types. Complete tables detailing all characteristics by each analysis method are 
available in Appendix A. 


Programmatic Factors 
Mentors and Host Schools 


Our first analysis was of programmatic factors that might influence whether teachers stay in the classroom, which 
revealed interesting but not marked differences. We found, for example, that while not statistically significant, 90.9% of 
the UTR and MASTER graduates who, during their clinical year, had a returning mentor—a teacher had served as a 
mentor previously and thus brought experience to the role—were teaching in NYC DOE schools as of the 2015-2016 
school year, compared to 85% of the graduates who had a new mentor. There was little difference (0.4%) in the teaching 
status of residents who did their residency at a new or returning host school. There was also a negligible difference 
(0.8%) in the percentage of residents teaching in NYC DOE schools of UTR residents compared to percentages of 
MASTER residents. (See Table 9.) 
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Table 9. Teaching Status Distribution by Mentor and Host School 


stopped StOP ped Currently Currently 
teaching teaching 3 or A A F ‘ 
— Sous teaching teaching Sig Chi- 
Teacher Characteristics within 2 yrs more yrs ates 
aes nets outside inside Square 
P P NYCDOE NYCDOE 
residency residency 
Returning Mentor (n=55) 3.6% 5.5% 0.0% 90.9% ng 
New Mentor (n=127) 5.5% 1.6% 7.9% 85.0% 
Returning Host School (n=120) 5.0% 2.5% 5.8% 86.7% - 
New Host School (n=62) 4.8% 3.2% 4.8% 87.1% 
UTR (n=150) 4.7% 3.3% 5.3% 86.7% Aa 
MASTER (n=32) 6.2% 0.0% 6.2% 87.5% 


Source: New Visions Data Warehouse, Teacher Certification Program Retention Data, accessed March 23, 2017. 


Subject Area 


Analyses of teaching status by subject showed that earth science (100%), followed by Special Education (93.5%), had 
the two highest rates of UTR and MASTER graduates teaching in NYC DOE schools; biology had the lowest rate 
(77.1%). Subject area was not, however, found to be a statistically significant variable in retention status. (See Table 10.) 


Table 10. Teaching Status Distribution by Subject Area 


Stopped Stopped Currently Currently 
Teacher Charsctrtioe | taser tenchng oof eee | ate | oat 
post-residency residency DOE DOE 
Special Education (n=62) 1.6% 1.6% 3.2% 93.5% 
Math (n=33) 6.1% 0.0% 9.1% 84.8% 
English (n=37) 2.7% 5.4% 5.4% 86.5% a 
Earth Science (n=4) 0.0% 0.0% 0.0% 100.0% 
Chemistry (n=11) 18.2% 0.0% 0.0% 81.8% 
Biology (n=35) 8.6% 5.7% 8.6% 77.1% 


Source: New Visions Data Warehouse, Teacher Certification Program Retention Data, accessed March 23, 2017. 


Resident Characteristics 


We also explored a number of resident characteristics or factors that may play a role in retention, again using Chi 
Square, ANOVA, and Logistical Regression analyses, as appropriate. In these analyses, there were some significant, if 


somewhat counterintuitive differences. 


Level of Education 


Our analyses suggest that the level of education with which a resident enters the program can affect their likelihood of 
remaining in the teaching profession—and remaining in the NYC DOE and likely in a higher-need school. Although 
retention rates were relatively high for all three groups, whether a resident’s highest level of education was a Bachelor’s, 
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Master’s, or Professional/Graduate/Doctorate Degree prior to beginning the residency program was found to have a 
statistically significant (0<.001) correlation with their teaching status, and there was an inverse relationship between level 
of education and retention inside the NYC DOE. Of the 140 residents whose highest level of education was a Bachelor’s 
degree, 90% were currently teaching inside the NYC DOE as of the 2015-2016 school year. Progressively fewer, or 
82.6% of the residents with a Master’s degree and 68.4% of the residents with a professional, graduate or doctorate 
degree were currently teaching inside the NYC DOE as of the 2015-2016 school year. (See Table 11.) 


Table 11. Teaching Status Distribution by Level of Education 


Stopped 
Stopped teaching 3 or Currently Currently Sig Chi- 
Teacher Characteristics teaching within more years teaching teaching a 
2 yrs post- post- outside inside q 
residency residency NYCDOE NYCDOE 
Prof/Grad/Doct Degree (n=19) 10.5% 10.5% 10.5% 68.4% eiiea 
Master’s Degree (n=23) 4.3% 0.0% 13.0% 82.6% df=6 p<.001 
Bachelor's Degree (n=1 40) 4.3% 2.1% 3.6% 90.0% 


Source: New Visions Data Warehouse, Teacher Certification Program Retention Data, accessed March 23, 2017. 


Of the 182 residents included in this analysis, 92.3% were retained as teachers, with 86.8% teaching in NYC DOE 
schools. Residents with a Bachelor’s degree were retained at higher than average rates in NYC DOE schools (90%) and 
at a higher rate when including residents teaching outside of NYC DOE (93.6%). Having a Master’s degree is associated 
with an above average retention rate (95.6%) if those retained outside of NYC DOE are included. See Appendix A for 
details on the average retention rates by teacher characteristic. 


The regression analysis showed that residents with a doctorate or professional degree equivalent have a statistically 
significant decreased likelihood (p=.020) of being retained in the classroom —a resident with a doctorate or the equivalent 
professional degree is .09 times less likely to be retained in the classroom. 


While our analysis indicates that a resident’s level of education prior to entering the program is linked to their retention, 
the data does not provide a reason why this relationship exists. UTR or MASTER graduates with advanced degrees may 
have additional employment opportunities available to them not available to someone with a Bachelor's degree, and 
those opportunities may have become increasingly available as the economy improved. Residents with advanced 
degrees may also have more work and/or life experiences and thus may be able to discern earlier in their teaching career 
whether the profession is a good fit. Compared to peers who began their residencies with a Bachelor's degree, 
residency graduates with advanced degrees may be older and have family commitments that would influence their 


decision to stay in the classroom or pursue more lucrative jobs. 


Gender and Ethnicity 


Although not statistically significant in the chi square analysis, the teaching status of UTR and MASTER graduates varied 
based on their ethnicity. All, or 100% of the Hispanic residents were currently teaching as of the 2015-2016 school year, 
with 95.8%, or all but one resident, teaching inside NYC DOE schools. Rates among Black and White residents were 
similar and also high, with 87% of both groups teaching in NYC DOE schools as of the 2015-2016 school year. 
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There were slight differences by gender, with a higher percentage of female graduates (88%) teaching in NYC DOE 


schools, compared to male graduates of the two programs (84.2%). (See Table 12.) 


Table 12. Teaching Status Distribution by Ethnicity and Gender 


Stopped Stopped Currently Currently 
teaching teaching 3 or teaching teaching 
within 2 yrs more years outside inside Sig Chi- 

Teacher Characteristics post-residency post-residency NYCDOE NYCDOE Square 
Multiracial (n=10) 0.0% 10.0% 10.0% 80.0% 
Asian (n=28) 8.7% 4.3% 4.3% 82.6% 
White (n=97) 41% 3.1% 6.2% 86.6% ns 
Hispanic (n=24) 0.0% 0.0% 4.2% 95.8% 
Black (n=23) 8.7% 0.0% 4.3% 87.0% 
Male (n=57) 8.8% 3.5% 3.5% 84.2% és 
Female (n=125) 3.2% 2.4% 6.4% 88.0% 


Source: New Visions Data Warehouse, Teacher Certification Program Retention Data, accessed March 28, 2017. 


Being Hispanic or female is associated with above average retention rates both within and outside of NYC DOE schools. 


Being Black or African-American is associated with a slightly higher rate of retention within the district, while being White 


is associated with a marginally higher than average retention rate when schools inside and outside of the NYC DOE are 


included. Being Asian or multiracial is associated with lower than average retention rates. (See Figures 12 and 13 below.) 


Figure 12. Retention Within or Outside NYC 
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Source: New Visions Data Warehouse, Teacher Certification Program Retention Data, accessed March 28, 2017. 


While being female was not found to be a significant variable in the Chi-Square analysis, being female was significant in 


the regression analysis. In the regression analysis, female residents have a statistically significant (0=.035) increased 


likelihood of being retained in the classroom. The odds of a female resident being retained are 6.7 times the overall odds 


of retention. 
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RETENTION AND RESIDENTS’ SENSE OF EFFICACY 


As part of our previous UTR survey analyses, we created a set of scales based on residents’ responses and a series of 


factor analyses. The scales included such constructs as general preparedness and efficacy, management, and 
engagement, discussed below. Appendix D includes all the scale constructs and survey questions that formed each one. 


Preparation, Practice, and Efficacy 


Growing evidence demonstrates that attrition is higher for those who enter the profession without adequate preparation. 
First-year teachers who feel they are well prepared for teaching are much more likely to plan to stay in teaching than 
those who feel poorly prepared.*? 


On the survey scales related to instruction, as shown by Figure 14, UTR graduates no longer teaching rated themselves 
lower in terms of preparation and instructional practices compared to residents who were still teaching. The largest gaps 
between those no longer teaching and those still teaching in NYC DOE schools were on the classroom management and 
the general classroom efficacy scales. UTR graduates no longer teaching had a rating of 2.5 on the classroom 
management scale and 2.4 on the general classroom efficacy scale, compared to ratings of 3.1 and 3.0 for residents 
currently teaching in NYC DOE schools. An ANOVA test found the differences in ratings on the classroom management 
scale between the three retention groups to be significant at the p=.031 level. This finding indicates that a resident’s 
perception of their preparedness to handle the challenges of classroom management is related to their likelinood of 
remaining in the classroom. 


Figure 14. Survey Scales Related to Teacher Efficacy 


3.3 
an 88 ve 30 341 og 31 3.1 3.0 3.1 3.0 3.0 3.0 
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Supporting Assessing Learning / Classroom General Instructional Overall Preparedness Overall Best 
Engagement Scale Understanding Scale Management Scale Efficacy Scale Scale Practices Scale 
No Longer Teaching (n=5) ™ Currently teaching outside NYC (n=4) ™ Currently teaching inside NYC (n=85) 


Source: REA Annual Teacher Surveys, 2011-2013. Means calculated on a 4-pt. scale, where 1=not at all prepared/confident, and 
4=very confident. 


48 karen J. DeAngeles, Andrew F. Wall, and Jing Che, “The Impact of Preservice Preparation and Early Career Support on Novice 
Teachers’ Career Intentions and Decisions,” Journal of Teacher Education 64, no. 4 (2013): 338-55. 
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Residents’ Assessments of School Environment during their Clinical Year 


School environment ratings imply that there may be a connection between UTR residents’ sense of a positive school 
climate during their clinical year and the likelinood that they will remain in the teaching profession. On the survey scales 
related to engagement and support, Figure 15, residents no longer teaching again had lower scores on all four scales— 
professional engagement, school climate, teacher support, and frequency of support—compared to residents who 
remained in the classroom. The largest difference between groups was on the school climate scale. Residents not 
currently teaching had an average rating of 2.5 (on a Likert scale, where 1=strongly disagree and 4=strongly agree), 
compared to an average rating among those currently teaching outside of NYC DOE of 3.5, and those currently teaching 
inside NYC DOE of 3.4. These differences between retention groups’ school climate ratings were statistically significant 
at the p=.026 level, based on the ANOVA test. 


Differences in individual survey items related to school climate and professional development, while not statistically 
significant, reveal some interesting distinctions between residents who are no longer teaching, those currently teaching 
outside of the NYC DOE, and those currently teaching inside NYC DOE schools. As illustrated in Figure 15, residents 
currently teaching inside or outside of NYC DOE schools had similar ratings—within one-tenth of a point on four out of 
the six survey items related to school climate and professional engagement. 


Figure 15. Survey Scales Related to Engagement and Support 
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No Longer Teaching (n=5) “Currently teaching outside NYC (n=4) | Currently teaching inside NYC (n=85) 


Source: REA Annual Teacher Surveys, 2011-2013. Means calculated on a 4-pt. scale, where 1=not at all prepared/confident, and 
4=very confident. 


Residents’ Thoughts on Retention 


As part of the survey completed at the end of their clinical year, residents are asked to reflect on their decision to 
become—and remain—teachers. One item asked UTR graduates if they would still become a teacher if they could start 
over again. On a five-point scale, where 1=definitely would become a teacher, 2=probably would, 8=chances even for 
and against, 4=probably wouldn’t, and 5=definitely wouldn’t become a teacher, interestingly, the five residents who left 
teaching, and the 85 residents who have remained in the classroom, within the NYC DOE, had the most positive ratings, 
at 1.2 and 1.4. 
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Those teachers currently teaching outside of NYC were a little more ambivalent, with ratings between “probably would” 
and even odds (M=2.3). All three groups of residents had the same response—a “Yes” — when asked if they planned to 
continue teaching. 


An additional set of survey items asked UTR residents about the relative importance of factors related to retention—or 
what might affect their decision to stay or leave. For those not teaching and those teaching within the NYC DOE, support 
factors—support from administrators, opportunities to collaborate, ongoing PD—were more important than they were for 
UTR graduates teaching outside the city. The widest gap in survey item scores between the three retention groups was 
in regard to the importance of a high salary. Of the three groups, those currently teaching outside NYC DOE indicated 
salary had a high importance, with an average rating of 3.5 (on a 4=point scale, where 1=not at all important and 
4=essential). Those currently teaching inside the NYC DOE indicated that a high salary was of moderate importance, with 
a rating of 3.1. A high salary was the least important to those residents who were no longer teaching, with an average 
rating of 2.8. These findings suggest that while salary may not have been an important factor in the decision of residents 
to leave the profession, it may be an influential factor for residents who choose to move to schools outside of the city, a 
little less important than time to collaborate, ongoing PD, and administrative support. (See Figure 16.) 


Residents in all three retention groups rated the importance of a positive school climate the highest, with each group 
having an average rating of 3.8 out of four for this item. A supportive administration received the second highest ratings 
from those no longer teaching (3.8) and those currently teaching inside NYC DOE schools (3.7). Those teaching outside 
of NYC DOE rated the importance of this category as 3.3. The importance of participating in decisions and time to 
collaborate were rated similarly across the three groups. 


This feedback about school climate corroborates the results of the survey scales and other research in the field: the 


consensus seems to be that school conditions have direct implications for teacher attitudes about their work and their 
decisions to remain at their school or in the profession. 


Figure 16. Factors Related to Retention 


3.8 3.7 be 3.8 3.8 3.8 
ea i 3.2 3.3 3.2 
Importance of Importance of Time Importance of Importance of Importance of High Imporatance of 
Supportive Admin to Collaborate Ongoing PD Positive Climate salary Participate in 
Decisions 
No Longer Teaching (n=5) = Currently teaching outside NYC (n=4) ™=Currently teaching inside NYC (n=85) 


Source: REA Annual Teacher Surveys, 2011-2013. Means calculated on a 4-pt. scale, where 1=not at all prepared/confident, and 
4=very confident. 
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2 
4 


ENTION AND INTIAL HIRING SCHOOL DEMOGRAPHICS 


School demographic data for 163 UTR graduates also suggests that an initial hiring school may play an important role in 


a graduate’s transition to becoming a full-time teacher of record, and one that has some bearing on placement and 
retention in high-need schools. 


The environment in hiring schools can vary widely, and in our analyses, we included a set of pertinent variables based on 
the available data. Table 13 below provides the mean value for the demographic variable categories for four teaching 
status categories: teachers who stopped teaching within three years, post-residency, and four years; those currently 
teaching outside the NYC DOE, and those teaching within the NYC DOE. 


Conducting an analysis of variance (ANOVA) test indicated that the percentage of white student population (0=.011) and 
the percentage of economic need (o=.003) at the initial hiring school are statistically significant indicators of residency 
graduates’ teaching status. This finding, however, may be contrary to what would be expected: the greater the 
percentage of white students at the initial hiring school, the more likely a resident was to stop teaching within two years 
post residency. Also contradicting expected outcomes, the lower the economic need index of a school, the more likely a 
resident was to stop teaching within two years post residency. 


These findings may point to other recent findings about retention trends in high-poverty schools, which indicate that 
teachers are leaving these schools not because they do not want to teach high-need students. Again, the research 
points to factors related to teaching conditions and climate, not to the students or the particular instructional challenges 
they may present.°° 


5° Richard M. Ingersoll, “Teacher Turnover and Teacher Shortages: An Organizational Analysis,” American Educational Research 
Journal 38, no. 3 (2001): 499-534. See also Susanna Loeb, Linda Darling-Hammond, and John Luczak, “How Teaching Conditions 
Predict Teacher Turnover,” Peabody Journal of Education 80, no. 3 (2005): 44-70. 
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Table 13. Initial Hiring School Demographic Variables and Teaching Status 


Initial Hiring School Stapped stopped Cutrenty” “Currently 
Banisaraphic Variables teaching within 2 teaching 3 or teaching teaching Sig ANOVA 
arp yrs post- more yrs post- outside inside NYC 9 


(mean) residency (n=7) residency (n=5) NYC (n=5) (n=146) 


Enrollment 917 780 438 735 ns 
% Asian Student Population 14% 9% 2% 10% ns 
% Black Student Population 35% 20% 48% 29% ns 
% Hispanic Student Population 30% 46% 47% 50% ns 
% White Student Population | 1(3,159)=3.8, p=.011 
% Students with Disabilities 17% 20% 20% 19% ns 
% Self-contained Students 2% 3% 2% 2% ns 
% ELL Students 5% 10% 12% 9% ns 
% Economic Need | 82% 70% 76% TO | H9,159)=4.8,p=.003 


2.4 


2.3 


2.2 


2.4 


Avg 8th Math ns 
Avg 8th ELA 2.6 2.4 2.3 2.4 ns 
Effective School Leadership 3.4 4.0 3.8 3.6 ns 
Rigorous Instruction 3.4 3.0 3.6 3.5 ns 
Trust 3.4 3.9 3.6 3.7 ns 
Supportive Environment 3.6 3.7 3.6 3.7 ns 
Collaborative Teachers 3.9 4.3 4.2 4.0 ns 
Strong Family Community Ties 3.3 3.2 3.6 3.4 ns 
Student Achievement 3.8 3.4 3.5 3.7 ns 
Safety 2.9 3.4 3.2 3.5 ns 


Source: NYCDOE School! Quality Reports and School Performance Dashboards 


Using an analysis of variance (ANOVA) test to examine initial hiring school demographic variables and those retained in or 
out of NYC DOE schools and those not retained yields similar results to the ANOVA analysis above. The percentage of 
white student population (0=.021) and the percentage economic need (p=.009) are statistically significant indicators of 
whether or not a resident is retained in the classroom. (See Table 14 below.) Again, our analysis indicated that the lower 
the percentage of white students and the higher the percentage of economic need at a resident’s initial hiring school, the 
more likely a resident is to be retained. Additionally, this analysis found the percentage of the Hispanic student population 
to be significant (o=.036) with the greater the percentage of Hispanic students at a resident’s initial hiring school, the 
more likely the resident was to be retained in the classroom. School safety ratings were nearing significance (p=.17) and 
suggest that a resident’s initial hiring school’s safety rating may be linked to a greater likelihood of a resident being 
retained. 


The regression analysis further supports the claim that school climate can affect teachers’ retention decisions, more so 
than students’ economic status. In our analysis, the economic need index of a graduate’s initial hiring school was found 
to only be marginally significant. The regression analysis found a statistically significant correlation (9o=.007) between the 
safety index of a UTR graduate’s initial hiring school and the likelihood of being retained. A one-unit increase in the Safety 
index of a graduate’s initial hiring school results in 3.6 times the overall odds of retention. The safety index is a composite 
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of the five safety related questions from the annual school survey the NYC DOE administered to parents, students and 
teachers. Findings from the survey, including the safety index figures, are available through the NYC DOE School Quality 
Guide. 


When considered in a separate logistic regression, not controlling for the other variables associated with school climate, 
safety was not a significant predictor of retention. It becomes significant only when also accounting for the effect of 
economic need (see above regression analysis results). However, these findings aren't necessarily counter-intuitive or 
contradicting the demographic analysis. High economic need is still a positive predictor of retention, but the regression 
results for safety run separately suggest that within high economic need schools, the level of safety may also contribute 
to retention. An extension of this finding could be that residents who were initially hired at well-functioning NYC high- 
need schools had a good model of how to succeed ("I see them doing it, so | can do it"), whereas those initially hired in 
lower need schools where this modeling was not available might not have been so convinced that they could really do 
what they needed to do to be effective in a higher need setting. 


Table 14. Initial Hiring School Demographic Variables and Retention 


Initial Hiring School % /mean Among ‘mean Among Sig diff Not OB REFERENCE (FROM 
Demographic Variables Those Not u pose peeaned Retained vs ABOVE) Anan: oe 
(mean) Retained (n=12) (in or out of NYC) Retained Those Retained in NYC 
(n=151) (n=146) 
Enrollment 860 725 ns 735 
% Asian Student Population 12% 10% ns 10% 
% Black Student Population 28% 29% ns 29% 
% Hispanic Student Population ae 
% White Student Population oe 
% Students with Disabilities 18% 19% ns 19% 
% Self-contained Students 2% 2% ns 2% 
% ELL Students 7% 9% ns 9% 
% Economic Need is 
Avg 8th Math 2.4 24 ns 24 
Avg 8th ELA 2.5 2.4 ns 2.4 
Effective School Leadership 3.6 3.6 ns 3.6 
Rigorous Instruction 3.2 3.5 ns 3.5 
Trust 3.6 3.7 ns 3.7 
Supportive Environment 3.7 3.7 ns 3.7 
Collaborative Teachers 4.0 4.0 ns 4.0 
Strong Family Community Ties 3.3 3.4 ns 3.4 
Student Achievement 3.6 3.7 ns 3.7 
See 3.1 3.5 se 3.5 


PO sicniticant 


Approaching significance 
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CORRELATIONS BETWEEN SCHOOL LEVEL FACTORS AND SELF-EFFICACY 


We also explored any links between school-level factors—economic need index, effective school leadership, rigorous 
instruction, trust, supportive environment, collaborative teachers, strong family community ties, student achievement, 
and safety—and residents’ perceptions of preparedness or self-efficacy. Our analysis of the full sample, as well as 
separate analyses of those teaching and those no longer teaching in NYC DOE, revealed no significant correlations 
between residents’ perceptions, as captured on the survey scales, of their general instructional self-efficacy or overall 
preparedness and any of the school-level factors. 
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Conclusions and 
Recommendations 


Our analyses indicate that UTR and MASTER graduates are staying in New York City classrooms, and that the New 
Visions-Hunter residency model has had a positive impact on retention—a key goal of the joint initiatives. Due to the 
timelines of the two projects, and the fact that later cohorts have fewer years in the classroom, we were not able to look 
too far beyond residents’ commitment period of four years —but, so far, signs are good. Not only are retention rates high, 
but mobility rates are low. The large majority of UTR and MASTER graduates— 84% and 88% respectively —are teaching 
in the NYC DOE and New Visions Charter schools they committed to when they became residents. Moreover, they are 
remaining in the classroom longer than other high school teachers in the NYC DOE, and longer than teachers who 
entered the profession through different preparation pathways. 


Our analyses also indicate that a number of factors that may influence teachers’ retention decisions and the likelihood 
that they will continue year after year. Personal characteristics, hiring school characteristics, residency experiences and 
confidence—the data suggest some links between these variables and retention and mobility. Retention rates, for 
example, are higher among Hispanic teachers trained through UTR and MASTER; rates are also higher among females 
than males. Both sets of findings may have something to do with teachers’ tendency to stay where faculty profiles are 
similar to their own, but we don’t know that for sure. Data also suggest that safety and salary may both play a role in 
teachers’ decisions to move, and move to schools outside the city, but other personal or economic factors, on which we 
have little data, likely also enter into the equation. 


The demographics of hiring schools, along with ratings for school culture and stability also seem to play a role in 
teachers’ decisions to stay or leave, though not always the role one might assume: an older, more experienced faculty or 
administrator may be a positive indicator of school stability that draws teachers to schools, but our data on the 
characteristics of schools UTR teachers move to indicates that schools with younger faculties and administrators, which 
may connote new ideas and fresh leadership, may be more attractive or a better fit for teachers early in their careers. 


The data also show that teachers are not leaving schools on the basis of a high economic need index or a lower 
percentage of white students—proxy measures for the perceived challenges or advantages at a school. These findings 
align with other research on retention trends, which indicates that teachers don’t leave the classroom because of 
challenges with students. That research, like ours, suggests that school climate plays greater role. 


The critical role of school climate and support is a consistent theme in our findings, from our retrospective analyses of 
resident survey data and our correlational analyses of program data. On annual surveys, completed as they were 
finishing their clinical year and looking ahead to entering classrooms as full-time teachers of record, residents in 
successive cohorts agreed that school climate would be the number one factor in decisions to stay or leave the 
profession. Perhaps more interesting, the residents who assigned lower survey ratings to school climate factors— 
including levels of support and collaboration—for their host schools were more likely to be among the program graduates 
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who later left their hiring schools. The data also show a correlation between residents’ sense of preparation and 
confidence, again as self-reported on year-end surveys, and the likelihood that they will remain in the classroom. 


These findings are important because they involve retention factors that project partners have some influence over, unlike 
hiring school demographics and culture, personal characteristics, life changes, or other post-residency factors over 
which they have less sway. 


The recommendations in some cases relate more to communication and awareness than programmatic changes. 


= In reviewing the factors and findings that pertain specifically to the residency and induction year, consider where 
best allocate resources to improve retention among future residents. 


= Explore ways to differentiate support—for residents, mentors, and host or induction year schools—based on data 
about personal or school characteristics that may play into teachers’ job satisfaction and eventual retention. 


= There will inevitably be variability in both host and placement schools. In residents’ orientation, training, and ongoing 
conversations between mentors and residents—and as residents begin their job search and make decisions—it may 
be helpful to talk explicitly about school environment, and what’s important, comfortable, and fungible for them, to 
ensure that they look at the challenges—and rewards—of teaching in urban environments and high-need schools 
with a clear eye. 


= While it may not be possible or advisable to create uniform host-school environments, candidly discuss the value of 
variation and the non-negotiables. 


= As residents and mentor/resident relationships evolve and mature, devote conversations or reviews to the level of 
satisfaction among groups the data suggest may be less likely to persevere in the profession—i.e., those with 
advanced degrees, males, etc. It may be helpful for mentors to reach out to previous mentors in similar situations or 


settings, or residents to previous residents, to discuss acclimation, application of unique skills sets, etc. 


= Continue to monitor the retention and mobility data for trends that may inform training and support. 
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Appendix A: Teaching Status Distribution by Teacher Characteristics 


Table A.1. Teaching Status Distribution by Teacher Characteristics 


Stopped 
Stopped teaching 
teaching 3 ormore | Currently | Currently 

within 2 yrs years teaching teaching 

post- post- outside inside Sig Chi- 
residency residency | NYCDOE | NYCDOE | Square _ 


Teacher Characteristics 


Returning Mentor (n=55) 3.6% 5.5% 0.0% 90.9% 


New Mentor (n=127) 5.5% 1.6% 7.9% 85.0% ic 
Returning Host School (n=120) 5.0% 2.5% 5.8% 86.7% bis 
New Host School (n=62) 4.8% 3.2% 4.8% 87.1% 
Multiracial (n=10) 0.0% 10.0% 10.0% 80.0% 
Asian (n=23) 8.7% 4.3% 4.3% 82.6% 
White (n=97) 4.1% 3.1% 6.2% 86.6% ns 
Hispanic (n=24) 0.0% 0.0% 4.2% 95.8% 
Black (n=23) 8.7% 0.0% 4.3% 87.0% 
Special Education (n=62) 1.6% 1.6% 3.2% 93.5% 
Math (n=33) 6.1% 0.0% 9.1% 84.8% 
English (n=37) 2.7% 5.4% 5.4% 86.5% fia 
Earth Science (n=4) 0.0% 0.0% 0.0% 100.0% 
Chemistry (n=11) 18.2% 0.0% 0.0% 81.8% 
Biology (n=35) 8.6% 5.7% 8.6% 77.1% 
Male (n=57) 8.8% 3.5% 3.5% 84.2% ie 
Female (n=125) 3.2% 2.4% 6.4% 88.0% 
UTR (n=150) 4.7% 3.3% 5.3% 86.7% ag 
MASTER (n=82) 6.2% 0.0% 6.2% 87.5% 
Overall (n=182) 4.9% 2.7% 5.5% 86.8% 
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UTR Supplemental Studies, #38: Special Education rf 


Figure A. 1. Teaching Status by Teacher Characteristic 


™ Stopped teaching within 2 yrs post-residency Stopped teaching 3 or more years post-residency 
“Currently teaching outside NYCDOE ™ Currently teaching inside NYCDOE 


Overall (n=182) am 796 
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Earth Science (n=4) 


English (n=37) 5% = 87% | 
Math (n=33) 

Special Education (n=62) 292%3%I 94% 
Black (n=23) 
Hispanic (n=24) 

White (n=97) 3% MM 87%} 

Asian (n=23) ESS SS som 


Multiracial (n=10) 

New Host School (n=62) 
Returning Host School (n=120) 
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Bachelor's Degree (n=140) 
Masters Degree (n=23) 
Prof/Grad/Doct Degree (n=19) 
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Table A.2. Percent Teaching Status Distribution by Teacher Characteristics 


Retained 

Not (in or out Retained 
Teacher Characteristics Retained of NYC) in NYC Notes 
Professional/Graduate/Doctorate Degree (n=19) 21.0% 78.9% 68.4% 
Biology (n=35) 14.3% 85.7% 77.1% 
Multiracial (n=10) 10.0% 90.0% 80.0% 
Chemistry (n=11) 18.2% 81.8% 81.8% 
Asian (n=28) 13.0% 86.9% 82.6% 
Masters Degree (n=23) 4.3% 95.6% 82.6% 
Male (n=57) 12.3% 87.7% 84.2% 
Math (n=33) 6.1% 93.9% 84.8% 
New Mentor (n=127) 71% 92.9% 85.0% 
English (n=37) 8.1% 91.9% 86.5% 
White (n=97) 7.2% 92.8% 86.6% 
UTR (n=150) 8.0% 92.0% 86.7% 
Returning Host School (n=120) 7.5% 92.5% 86.7% 
Overall (n=182) 7.6% 92.3% 86.8% 
Black (n=23) 8.7% 91.3% 87.0% 
New Host School (n=62) 8.0% 91.9% 87.1% 
MASTER (n=82) 6.2% 93.7% 87.5% Data for shorter duration 
Female (n=125) 5.6% 
Bachelor's Degree (n=140) 6.4% 
Returning Mentor (n=55) 9.1% 90.9% 90.9% 
Special Education (n=62) 3.2% 
Hispanic (n=24) 0.0% 
Earth Science (n=4) 0.0% 100.0% 100.0% Small sample 


OE -av ve average in both retention categories 


=above average in one retention category (or in both but limited data) 
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Appendix B: Safety Logistic Regressions 


Table B.1. Logistic Regression for Safety 


Variables in the Equation 


B S.E. Wald df Sig. Exp(B) 
Step Female 1.898 .900 4,448 1 0.35 6.672 
1° Professional -2.398 1.028 5.443 1 0.20 0.91 
Doctorate 
Degree 
Economic 5.307 3.227 2.706 1 .100 201.770 
Need Index 
Safety 1.276 A477 7.151 1 .007 3.583 
Constant -5.083 2.925 3.020 1 .082 .006 


a. Variable(s) entered on step 1: Female, Professional Doctorate Degree, Economic Need Index, Safety. 


Table B fou.2. Logistic Regression for Overall Safety 
Variables in the Equation 


B S.E. Wald df Sig. Exp(B) 
Step Safety 457 333 1.880 1 .170 1.579 
1° Constant 1.028 1.090 890 1 346 2.796 


a. Variable(s) entered on step 1: Safety. 
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Appendix C: Teacher Survey Data Scores 


Table C. 1. Teacher Survey Data Scores 


Currently 
No Longer Currently teaching teaching inside Sig 
Teaching (n=5) | outside NYC (n=4) NYC (n=85) ANOVA 
Supporting Engagement Scale 3.0 3.3 3.3 ns 
Assessing Learning / 
Understanding Scale oe on = bi 


Classroom Management Scale 


General Instructional Efficacy 


2.4 3.1 3.0 ns 

Scale 

Scales _ |_Overall Preparedness Scale 2.6 3.1 3.0 ns 
Overall Best Practices Scale 3.0 3.0 3.3 ns 
Professional Engagement Scale 2.7 2.9 3.1 ns 
School Climate Scale 
Teacher Support Scale Zi 3.2 3.3 ns 
Frequency of Support / PLC 27 34 30 S 
Scale 
Start Over Would Become a 42 23 1A sia 
Teacher 
Plan to Continue Teaching 1.0 1.0 1.0 ns 
Plan to Continue Teaching at 15 10 14 As 
this School 
Importance of Supportive 3.8 33 37 im 

ran Admin 
Individual = Famel 

terns mportance of Time to 34 3.3 ee a 
Collaborate 
Importance of Ongoing PD 3.2 3.0 3.4 ns 
Importance of Positive Climate 3.8 3.8 3.8 ns 
Importance of High salary 2.8 3.5 3.1 ns 
Importance of Participate in 3.9 33 3.2 i” 
Decisions 


SS Sigriticat 


Lower than other two groups 
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Appendix D: Survey Scale Composition 


NOTE: Only items that were represented in all three waves of the survey (2011, 2012, and 2013) were included in 


the scales to allow for a valid longitudinal comparison. 


Supporting Engagement Scale 


ow prepared are you to motivate and engage students 


ow often do you provide opportunities for students to process information by writing or talking 


t 


ow often do you ask questions that push students toward higher-level thinking 


rz 


ow often do you create opportunities for students to work both independently and collaboratively 


Assessing Learning / Understanding Scale 

How prepared are you to use data to inform instruction 

How prepared are you to create assessments that measure short-term growth 
How prepared are you to create assessments that measure long-term growth 


ow often do you check for understanding during lessons or use in the moment assessments 


ow often do you check for understanding at the end of the lesson 


How often do you involve students in assessing their own learning 


Classroom Management Scale 


ow prepared are you to handle a range of classroom management or discipline situations 


ow often do you maintain class routines and ensure that discipline problems don't interfere with lessons 


+ 


ow often do you keep students on task from bell to bell 


General Instructional Efficacy Scale 
How prepared are you to teach your subject matter 


How prepared are you to improve student achievement 


NOTE: Items in the scales listed above are also represented in the general / overall scales below. 


Overall Preparedness Scale 
How prepared are you to: 
Teach your subject matter 


Handle a range of classroom management or discipline situations 
Motivate and engage students 

Use data to inform instruction 

Differentiate Instruction based on students' needs 

Create assessments that measure short-term growth 

Create assessments that measure long-term growth 

Improve student achievement 


Overall Best Practices Scale 
In your teaching, how often do you: 
Check for understanding during lessons or use "in the moment" assessments 
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Provide opportunities for students to process information by writing or talking 

Assess individual student needs 

Involve students in assessing their own learning 

Ask questions that push students toward higher-level thinking 

Create opportunities for students to work both independently and collaboratively 
Maintain class routines and ensure that discipline problems don't interfere with lessons 
Keep students on task from bell to bell 

Check for understanding at the end of the lesson 


Professional Engagement Scale 

How often do you engage in the following: 

Collaborate with other teachers on lessons and classroom planning 
Participate in departmental or grade-level meetings 

Collaborate with colleagues on school-wide instructional decisions 


School Climate Scale 

School leaders communicate a clear vision for this school 

School leaders encourage open communication on important school issues 

Curriculum, instruction, and assessment are aligned within and across grade levels 
School leaders invite teachers to play a meaningful role in setting goals and making important decisions 
There is a great deal of collaboration among staff members 

Most of my colleagues share my beliefs and values about the central mission of the school 
Teachers are recognized for their efforts 

Teachers at this school like being here 

Teachers are reflective and think critically about their practice 

Teachers see themselves as members of a professional learning community 


Resident Support Scale 

To what extent do you feel supported by: 
Your principal 

Assistant principals or other administrators 
Other teachers; Outside groups 


Frequency of Support Scale 

How often is the following kind of support made available to you: 

Common planning time with teachers in your subject 

Time for inquiry teams, reflective seminars, PLCs, critical friends 

Coaching from your UTR site director 

Regular supportive communication with your principal or other administrators 
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